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Key messages 



•y- The National Reporting System (NRS) for adult English language, literacy and numeracy has 
influenced the content and emphasis of literacy and numeracy curricula nationally, and played a 
role in the literacy and numeracy aspects of training packages. 

•y- Those interviewed for this study all agreed it was time for the NRS to be reviewed. 

•y" It was felt that a review would provide an opportunity to: 

♦ streamline the conceptual framework 

♦ develop a set of user-friendly support materials 

♦ revise the rules for reporting within Commonwealth programs 

♦ consider how best to provide adequate and ongoing professional development, with an 
opportunity to build national ownership of outcomes 

♦ further explore ideas for broader applications. 
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Executive summary 



Funded joindy by the Australian National Training Authority (ANTA), and the then Department 
of Employment, Education and Training, the National Reporting System (NRS) was intended as: 

... a mechanism for reporting the outcomes of adult English language literacy and numeracy 
provision, in the vocational education and training system, in labour market programs and in 
the adult community education sector. (ANTA 1995) 

It has long been used in other ways as well — as a means of evaluating the content and emphasis of 
adult basic education curricula, as a framework for the development of curriculum and assessment 
materials, and in the consideration of the language, literacy and numeracy requirements of 
training packages. Elowever, its uptake by literacy and numeracy practitioners has been confined 
mainly to those required to use it as a reporting tool for the Commonwealth-funded Workplace 
English Language and Literacy (WELL) program, and the Language Literacy and Numeracy 
Program (LLNP). 



Project aim 

‘The NRS Future Directions’ project was designed to provide initial insights into how the NRS is 
currently being used and how it is perceived by a group of people familiar with its concepts and use. 
This exercise was seen as a necessary precursor to any changes that might be made, and was not 
meant to be a full-scale review in its own right. 

The methodology involved: 

■y- a review of relevant literature 

•y" monitoring of relevant sessions at the Australian Council for Adult Literacy (ACAL) national 
conference 

•y" detailed interviews with a core group of 50 interested parties, some recommended and others 
selected at random. A further 30 people provided input during less structured discussions. 

During the study, the NRS was considered as an instrument for describing language, literacy and 
numeracy competence; as a reporting mechanism within the WELL and LLNP, and as a 
mechanism for non-reporting applications. Information was gathered about its strengths and 
weaknesses and the types and degrees of support required for effective application for different 
purposes. Interviewees were asked to identify key issues and possible future directions for the NRS, 
as well as current and future priorities in regard to adult literacy and numeracy, and in relation to 
education and training generally. 



Key findings 

The NRS as an instrument for describing language, literacy and numeracy 

Responses suggest that the NRS has built a strong following amongst those with the background 
knowledge and opportunity to become expert users. Expert practitioners reported finding the NRS 
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an extremely useful tool for thinking about a student’s strengths and weaknesses, for planning, for 
designing appropriate assessment materials, and for curriculum development and evaluation. 

Academics interviewed generally considered the conceptual underpinnings sound. However, they, 
and most practitioners, suggested that the conceptual base be streamlined to produce a more 
straightforward instrument. Almost all agreed that there was no need to start again from first 
principles, but that the process should aim, as one interviewee explained, to make the NRS ‘more 
elegant’. While a revised NRS would incorporate new knowledge and understandings about literacy 
and numeracy, and reflect practitioners’ experience after eight years of application, it should 
maintain the tool’s current ability to capture the richness and complexity of language, literacy and 
numeracy across contexts. 

The NRS as a tool for reporting within the Workplace English Language and 
Literacy, and Language, Literacy and Numeracy programs 

There is a need to separate the NRS instrument from the rules and logistics associated with 
reporting to funding bodies. Practitioners of the Language, Literacy and Numeracy Program 
identified some rules that appeared to go against the principles of the NRS and had issues with the 
logistics and time involved in reporting to funding bodies. However, most stated that the NRS 
framework had influenced the way they thought about their students’ skill levels, the way they 
taught and the way they assessed. However, some WELL practitioners were more critical of the 
usefulness of the tool, questioning the effort that went into learning about the NRS, its limited 
application within the program, and its lack of real impact on teachers or learners. 

Support required for effective application as a reporting tool 

Interviewees identified four essential components of an overall ‘package’ to support its use as a 
formal reporting tool. These were: 

■y- a level of background knowledge of language, literacy and numeracy sufficient to be able to 
appreciate the conceptual aspects of the NRS and interpret them appropriately 

•T regular opportunities for interactive professional development activities 

•T accessible, relevant support materials 

-y" time. 

If any one of these were missing, many felt it would be hard to sustain the NRS (current or revised) 
in the future. 

Non-reporting applications 

This survey confirmed that the NRS continues to be ‘more than a reporting tool’. For example, it 
found strong support for the NRS as a tool for reflective practice, and widespread application as a 
framework for developing curriculum and assessment materials. Some interviewees had developed 
simplified versions of the NRS — in effect, its key concepts only — as part of processes to raise 
language, literacy and numeracy awareness in industry. In the vocational education and training 
(VET) sector, training package developers are required to use the NRS as part of their design brief, 
although interviewees familiar with the results of this application felt that this directive had 
operated with varying degrees of success. 



Plotting future directions 

In considering the future of the NRS, several possible goals were discussed: 
■v" to maintain the status quo 
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V to make the current NRS more user-friendly and accessible 

V to increase uptake of a current or revised NRS in order to improve practice 

V to push for national uptake as a reporting mechanism 

■v- to reframe and reposition the NRS as a general language, literacy and numeracy framework 

V to link the NRS directly into the mainstream of education and training. 

No interviewee wanted to maintain the status quo. There was strong support for a review and 
revision of the NRS to make it more user-friendly, in the hope that this would increase take-up by 
other practitioners. There was little support for the idea of simplifying the NRS for use as a 
reporting tool by non-practitioners. However, those who were already using the NRS for non- 
reporting purposes supported the idea of reinventing it as a general language, literacy and numeracy 
framework which would raise awareness of key concepts outside the small language, literacy and 
numeracy field. Some interviewees could also see a specific role for a simple, conceptually elegant 
framework, based on the current NRS, as a key tool for addressing language, literacy and numeracy 
issues within revised training packages. 

There was little interest in (and indeed some resistance to) any push for adoption of the NRS as a 
reporting system at state and territory level. 



Conclusions 

There are indications that the NRS is more firmly entrenched within the adult language, literacy 
and numeracy context than may be immediately apparent. Importantly, the NRS appears to have a 
base of support amongst influential people in the field, and has become an integral part of the 
philosophy and methodology of a number of expert — and highly regarded — practitioners. 

Although once recognised as ahead of its time, the instrument now needs updating, particularly in 
light of developments in information and communication technology. It also needs streamlining 
conceptually. Thus, it is timely to conduct a formal review focused on ways of improving a useful 
tool. If this review is specifically designed to facilitate constructive exploration and sharing of ideas 
across the language, literacy and numeracy field, it could also contribute to general professional 
development, and build a broad sense of ownership of the revised version. There is no point in 
going ahead with the development of new applications for the NRS until a review has been 
completed. However, discussion of future directions and possibilities should be considered by 
professionals in the field. 

If broader uptake is envisaged, face-to-face professional development opportunities for both new 
and experienced users should be envisaged and resourced as an integral part of any future NRS 
‘package’. A new look, streamlined NRS will undoubtedly be more user-friendly, but it will never 
be easy to grasp and use as a reporting tool. Its very strength in managing diverse aspects of 
language, literacy and numeracy in context provides challenges for new users that can be more 
effectively addressed through interactive professional development than via print-based manuals. 
More experienced users also benefit from the chance to hone their understandings through 
opportunities to share ideas and insights. 

Although the NRS has gradually become interwoven into the fabric of adult literacy, it does not 
have a high profile, and has not become an integral part of mainstream adult education, despite 
recognition of the need to increase literacy and numeracy understanding and skills within the VET 
sector. However, the concepts encapsulated within the NRS could be extremely useful in this 
regard, and could be promoted as part of the revision of training packages. Nevertheless, even a 
revitalised NRS is likely to present as ‘too much information’. A literacy and numeracy framework. 
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based on key NRS concepts, would provide an effective awareness-raising tool, be easier to promote, 
and be more likely to be adopted. 

Since its implementation, the NRS has been ‘more than a reporting tool’. Any revision should 
involve the exploration of the potential for a new framework for thinking, analysing and discussing 
language, literacy and numeracy in multiple contexts. Despite some people’s reservations about the 
term, a framework provides the scaffolding upon which many things can be built. The NRS has 
already proved to be flexible enough to become the support structure for multiple purposes. 
Developing a new entity, and calling it a literacy and numeracy framework, would signal a formal 
recognition of multiple applications which has not been possible with the NRS. 

The framework could be developed in several formats for various purposes and audiences. A full 
(revised) version would still offer the richness of the original NRS, while a simplified version might 
introduce key concepts about literacy and numeracy to non-specialist audiences, and thus become 
an important tool in the push to improve general workplace understanding of language, literacy and 
numeracy. Even though there might be different versions of the framework, they would all be built 
on the same foundations, using the same terms and levels. Besides maintaining consistency, this 
would provide a link between specialists, workplace trainers and industry. 



Recommendations 

•y" Continue to use the NRS as a key tool in reporting on adult literacy and numeracy performance. 

■v- Review and revise the current NRS. Develop a package incorporating the instrument itself, a 
revised set of rules for use for reporting within Commonwealth programs, a set of user-friendly 
support materials, and professional development activities designed for specialist users. Design 
the review process as a vehicle for professional development in its own right. Use it to further 
explore a range of options for future directions for the NRS and build a sense of ownership of 
the outcomes. 

•y" Explore the potential to use the key concepts and structure underpinning a revised NRS as the 
basis for the development of a literacy and numeracy framework (with a new name) which could 
be presented in different formats and levels of complexity for a range of audiences and purposes. 
This option could also be incorporated into the proposed review. 

■v" Review the arrangement for provision of NRS training and ongoing mentoring within the 
Language, Literacy and Numeracy and Workplace English Language and Literacy programs. 
Consider ways of funding and delivering ongoing language, literacy and numeracy training to 
other target groups, for example, through Industry Training Councils and professional bodies. 

^ If a new language, literacy and numeracy framework is developed, promote it through 

interactive professional development programs tailored for different groups to enhance their 
understanding of language, literacy and numeracy. 

■v- Devise a strategy to ensure that a new framework (or failing that, the revised NRS) plays a key 
role in determining the language, literacy and numeracy aspects of revised training packages. 
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Introduction 



Background 

Funded joindy by the Australian National Training Authority (ANTA), and the then Department 
of Employment, Education and Training, the National Reporting System (NRS) was intended as: 

... a mechanism for reporting the outcomes of adult English language literacy and numeracy 
provision, in the vocational education and training system, in labour market programs and in 
the adult community education sector. (Coates et al. 1995) 

It was designed to provide: 

... a national framework and common language that would act as a reference point for 
providers, facilitate student pathways and contribute to national information systems for 
accountability and planning purposes. (Cumming 1997, p.7) 

A range of prospective purposes and user groups was envisaged. The original document suggests 
that the NRS could be used by: 

T students who need a report describing their achievements 

T curriculum developers who need consistent statements for the development of courses for 
accreditation 

T program providers who need to be able to report to funding bodies on the achievement of 
individuals in terms of language, literacy and numeracy outcomes 

T Commonwealth Employment Service staff who need consistent and commonly used 
reporting information from providers when considering clients’ English language, literacy 
and numeracy development and future training needs 

•v* funding authorities who need to receive reports on student outcomes as one source of data 
to account for expenditure 

T industry and training personnel who need a common reference point for describing English 
language, literacy and numeracy within competency standards and training programs. 

(Coates et al. 1995, p.l) 

Since its publication, the NRS has been used as a means of reporting on adult language, literacy and 
numeracy within the Workplace English Language and Literacy (WELL) program, and the 
Language Literacy and Numeracy Program (LLNP). It has also been used to evaluate the content 
and emphasis of adult basic education curricula, and for assessing the language, literacy and 
numeracy requirements of training packages. However, the NRS has never become a genuinely 
national reporting framework. 

The NRS was primarily created for use by specialist language, literacy and numeracy practitioners, 
but anecdotal evidence suggests that few use it outside WELL and LLNP, with some professionals 
reportedly unwilling to expend the effort required to work out what the NRS is about, or how it 
might apply to their practice. This has led to calls for its simplification. There have also been 
criticisms that the NRS is too complex and difficult for use by those who do not have specialist 
training, and suggestions that it should be simplified to make it more accessible to workplace trainers. 
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On behalf of the Department of Education, Science and Training, the National Centre for 
Vocational Education Research (NCVER) has undertaken to conduct this first phase of an 
exploration to identify possible future directions for the NRS. 



Project aim 

‘The NRS Future Directions’ project was designed to provide initial insights into how the NRS is 
currently being used, how it is perceived by a group of people familiar with its concepts and use, 
and its potential for adaptation to a broader range of applications. 



Scope 

It was not intended that the project be a full review of the NRS, or a detailed examination of its 
various applications, such as within WELL and LLNP or in the design of training packages. 

The project was designed as a scoping exercise to seek feedback to inform decisions concerning next 
steps. Thus it was intended to: 

•T provide information on key people’s perceptions of the strengths and weaknesses of the NRS 

V highlight issues that might have a bearing on future applications 

V identify options that might be appropriate for more in-depth exploration and development. 

Methodology 

The project was conducted in three stages. 

Stage 1: Project planning 

V background reading of NRS-related documents 
A literature search and review 

V initial discussions with several government officers, practitioners and academics 

•y" attendance at an introductory workshop designed for practitioners new to the NRS 

V finalisation of a list of those to be consulted, and initial contact by telephone or email to invite 
their participation. 

Stage two: Targeted consultation 

V development of draft questions, trial and modification 

V conduct of face-to-face or telephone interviews with identified participants 
•y" analysis of data 

•y" development of a draft report highlighting current strengths and key issues as identified by 
interviewees and containing options for change. 

Stage three: Testing of conclusions and recommendations 

V monitoring of NRS-related sessions at the Australian Council for Adult Literacy 2004 conference 
•y" informal discussions about the NRS and proposed options with conference attendees 

V development of final report. 
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Sources of information 

Some 50 people provided input through formal interviews conducted in person or by telephone. 
Interviewees included: 

•y- academic experts in the field of adult literacy and numeracy 

•y" practitioners experienced in the use of the NRS (as well as some who had received NRS training 
but did not use it) 

■y- members of the Australian Council for Adult Learning and other professional bodies 
•y- representatives of federal and state training authorities 
•y - representatives from industry bodies 

■y" members of the initial NRS development team and academic reference groups 

~ y - moderators and verifiers operating in the Language, Literacy and Numeracy Program. 

Informal discussions were also conducted with another 40 people, including adult and community 
education (ACE) practitioners attending an introductory NRS workshop, and with many attendees 
at the Australian Council for Adult Literacy 2004 national conference. Conference sessions focusing 
on NRS applications and on LLNP and WELL were also monitored. 

Report structure 

Chapter 1 introduces the project’s aims, scope and stages and describes the methodology employed. 

Chapter 2 provides a brief background to the NRS for those unfamiliar with its history and 
features. 

Chapter 3 outlines the project methodology. 

Chapter 4 provides an overview of findings, with examples of feedback from interviewees. 

Chapter 5 discusses possible goals for change, and potential strategies to achieve them. 

Chapter 6 outlines conclusions and recommendations. 
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The NRS 



Origins 

As noted in the introduction to this report, the establishment of the NRS was jointly funded in 
1994 by the then Department of Employment, Education and Training and the Australian 
National Training Authority to support the development of a system to report the outcomes of 
adult English language, literacy and numeracy programs in VET, adult and community education 
and the workplace. 

The NRS was developed by the Adult, Community and Further Education Division of the Office 
of Training and Further Education in Victoria. 

Although it drew heavily on the work of others, the NRS was something completely new, and it was 
developed in a context of intense philosophical and political debate (Coates 1995, 1996). Indeed, 
some respondents to this project were amazed that the NRS had ever emerged, given that, ‘no one 
theory of language or learning was, as yet, sufficiently developed or uncontested to provide the basis 
for the project’ (Coates et al. 1995, p.3). A set of guiding principles and minimum specifications 
was developed based on extensive consultation with potential users and academics, whereby it was 
determined that the NRS should: 

V satisfy a variety of purposes and the requirements of a range of prospective users 
•y" reflect and promote good educational practice 
•T be fair to participants, valid and reliable 
•y" be functional in practice. 

The NRS was designed to explain and track progress along a continuum, and developers aimed to 
base it as closely as possible ‘on a model reflecting real life performance’ which aimed to be inclusive 
of a range of theories, philosophies and curriculum approaches (Coates et al. 1995, pp. 1 — 4). 



The instrument 

The instrument to emerge from the NRS development process was no simplistic measuring tool. 
Using a five-point scale based on that used in the International Adult Literacy Survey, the NRS 
went further than the international survey in attempting to address the complexity of adult 
language, literacy and numeracy development. This involved a recognition of the importance of 
context, and the potential for an individual to perform at different levels of competence, depending 
on mode, aspect, and degrees of support. The NRS also broke new ground by incorporating a 
measure of an individual’s learning strategies. 
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Box 1 : Key features of the NRS 



The NRS incorporates indicators of competence in five modes: reading, writing, oral communication, 
learning strategies and numeracy. 

An individual's performance in each mode is measured across 5 levels, taking into account factors such as 
task and text complexity, numeracy complexity, familiarity of context, learning strategies employed and the 
nature of support provided. 

Different aspects of communication are also recognised through six aspects of communication: procedural, 
technical, personal, cooperative, systems and public. (Coates et al. 1995) 



More than a reporting system 

The NRS pilot program was trialled and formally evaluated in 1997. 

The evaluation found that the NRS provided ‘a valid framework for reporting adult language, 
literacy and numeracy outcomes across a range of sectors, contexts and learners’ (Cumming 1997). 

Interestingly, even in these early days of its use, the evaluation also found that the NRS was being 
used in non-reporting applications, for example, in curriculum development and review; program, 
regional, state and system planning and reporting; and the integration of language, literacy and 
numeracy in vocational training (Cumming 1997, pp.8— 9). Although the potential for such 
application had been foreshadowed to some extent by the NRS development team, the discovery 
that it was actually happening informed the choice of title for the evaluation report, More than a 
reporting system. 

It was originally intended that the NRS be used to report outcomes of the Special Intervention 
Program and the Adult Migrant Education Program (AMEP). A change of government at the end 
of 1996, and a restructuring of labour market programs saw the Special Intervention Program 
discontinued. At the same time, a decision was made within the Adult Migrant Education Program 
not to adopt the NRS, but to continue to report against the Certificate in Spoken and Written 
English because it was seen as ‘a tried and tested system which is widely accepted by teachers, service 
provider management and policy making and funding organisations’ (Cumming 1997, pp.68— 9). 

With the change of political direction, the role of the NRS became unclear for a time and, due to 
lack of support from some states, it was never officially ratified by the state education ministers. 
Although it was eventually decided that the NRS would be used to report against the new 
Commonwealth Literacy and Numeracy Training program, the professional development originally 
planned to launch and support NRS implementation did not occur. One member of the 
development team noted that this combination of events meant that, ‘people were forced to use the 
NRS, but without rigorous professional development it never achieved its potential’. 

Over the past eight years, the NRS has been used to report on two Commonwealth-funded 
programs — Workplace English Language and Literacy (WELL), and the Language Literacy and 
Numeracy Program (LLNP). It has also demonstrated its value in non-reporting fields, for example: 

■v- Most adult basic education curricula used in Australia have been mapped against the NRS. 

•y" From 1997, the NRS was used as a foundation document for the development of advice to 
training package developers about how to incorporate language, literacy and numeracy skills 
within the training packages. This policy coined the phrase ‘built in not bolted on’, which has 
since become synonymous with all approaches to integrated literacy provision. 

Although there has not been any formal updating of the original NRS publication, support 
materials have continued to be produced. In 1999, the NRS document and case studies 
demonstrating education and training applications were placed online, and in 2001, a set of 
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industry-validated workplace sample activities was developed to augment the existing sample 
activities of the NRS. Many of the professional development resources associated with the NRS are 
posted on the Department of Education, Science and Training Literacynet website 
<http://www.dest.gov.au/literacynet>. 
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Project methodology 



Major areas of investigation 

Prior to considering the potential future uses of the National Reporting System, it was important to 
seek feedback on how it is currently being used and to gather perceptions about its effectiveness. 

During the project the NRS was considered: 

V as an instrument for describing language, literacy and numeracy competence 
^ as a reporting mechanism within WELL and LLNP 
^ as a tool for non-reporting applications. 

Information was gathered about the strengths and weaknesses of the NRS in each situation, and 
about types and degrees of support required for effective application for different purposes. 
Interviewees were asked to identify what they saw as key issues and possible future directions for the 
NRS. They were also asked about the context within which they operate, and what they saw as key 
needs and priorities in regard to adult literacy and numeracy, and in relation to adult education and 
training generally. 



Project key questions 

Formal interviews sought to discover answers to the following key questions under each heading: 

V The NRS as an instrument for describing language, literacy and numeracy competence 

♦ Do people accept the theoretical construct? 

♦ What do people perceive as the strengths and weaknesses of the tool, conceptually and in 
practice? 

♦ Is the NRS seen to be applicable in a range of contexts and with different individuals and 
groups? 

-y" The NRS as a reporting mechanism within WELL and LLNP 

♦ Is the NRS perceived to be effective as a reporting tool? 

♦ How do the current reporting rules impact on funding bodies, providers, teachers and learners? 

•y" The NRS as a mechanism for non-reporting applications 

♦ How is the NRS being used and by whom? 

♦ What options do users envisage for the future? 

♦ What are their responses to a range of suggestions? 

♦ What can be learnt from overseas experience? 

■v- Requirements for effective application 

♦ What do new users need in order to be able to understand and apply the NRS effectively for 
different purposes (for example, background knowledge and experience, professional 
development, support materials)? 

♦ What do experienced users need? 

♦ How far are these needs currently being met? 
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•y" The context 

♦ What are the key issues and trends relevant to adult literacy and numeracy provision? 

♦ What political drivers and opportunities might provide an opportunity to increase the impact 
oftheNRS? 

The full question list (see appendix 2) was framed to provide detailed responses in each area; the 
questions provided a general framework for each interview. However, given the range of experience 
and background knowledge of the interviewees, it was not envisaged that each person would be 
asked to cover all aspects. 



The interviewees 

The interviewees who contributed to this project possessed a wealth of knowledge about adult 
language, literacy and numeracy development. Although it was not intended that this initial survey 
be large enough to encompass a statistically valid representative sample, efforts were made to consult 
with a broad cross-section including: 

•T those who had been involved in the NRS conceptualisation process, had been part of the 
development team, and/or who had developed support materials following implementation 

•T practitioners 

•y" federal and state policy-makers and funders responsible for adult literacy and numeracy programs 
•y" industry representative bodies such as trade unions and Industry Training Councils. 

•y" academics from the adult literacy and numeracy field. 

In relation to consultation with practitioners, the majority had used the NRS extensively, but some 
were new users, and a few were familiar with the NRS, but did not use it. NRS users came from five 
states and one territory, and were experienced in the application of the NRS for various groups such 
as native English speakers, Indigenous people and migrants from non-English speaking 
backgrounds, the deaf, and people with disabilities. Some were closely associated with government- 
funded programs in VET or ACE. Others were employed in industry, some in technical and further 
education (TAFE), and others as not-for-profit or private providers. 

See appendix 1 for a list of contributors. 



Interview process 

Sixty-six individuals were approached to provide formal input. All those who were contactable were 
pleased to participate, and fifty interviews were conducted during July and August of 2004. It was 
occasionally possible to schedule a face-to-face meeting, but most interviews were conducted by 
telephone, and took between 45 and 60 minutes. 

There was some variation in the content and focus of each interview. Given that the focus was on 
qualitative rather than quantitative information, and because the interviewees came from diverse 
backgrounds, it was accepted that not everyone would be in a position to comment on all areas. 
Interviewees were also given the opportunity to identify those aspects they considered most 
important if they had not already been mentioned. 
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Findings 



In exploring current perceptions of, and uses for, the NRS, it was considered both as an instrument 
for describing language, literacy and numeracy competence, and in its capacity as a formal tool for 
reporting. This made it possible to separate discussion of its conceptual underpinnings from 
perceptions of reporting processes and rules. Non-reporting applications were also considered, as 
were interviewees’ perceptions of the qualifications, training and resources required to utilise the 
NRS effectively in both reporting and non-reporting functions (see figure 1). 

This section outlines the major findings under the four main headings used: 

■v" the NRS as an instrument for describing language, literacy and numeracy competence 

■v" the NRS as a reporting tool 

■v" non-reporting applications 

■v" requirements for the effective use of the NRS. 

Quotes from interviewees have been selected as indicative of the range of responses to each question. 



Figure 1 : Project focus areas 
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The NRS as an instrument for describing language, literacy 
and numeracy competence 

Perception of rationale 

Those interviewees who had been involved in the development of the NRS had first-hand 
knowledge of the rationale behind its development. Amongst this group there was general 
acknowledgement of the need for a mechanism to describe language, literacy and numeracy 
competence, and of the need for accountability. Comments included: 

Given the vast complexity of stakeholders and curriculum applications, there was a need for a 
kind of operating system. 

We knew programs were getting good outcomes, but we couldn’t show it. 

The Commonwealth wanted an empirical basis for demonstrating progress. It had to fit the 
ANTA framework of the time and go into the mainstream and relate to competency-based 
training. 

The original purpose of the NRS was as a national reference point. We still need this. It 
means you can say, ‘This learner is here’ — regardless of the curriculum. 

It is important for people to have a record of progress. You can’t do literacy without measures 
of how we are going — some kind of accountability. 

When asked what they saw as the purpose of the NRS, practitioners who had not been part of the 
development had a range of responses. 

The NRS is for reporting progress. It supports people who are moving around. 

We have had huge discussions about its purpose — we decided you need benchmarks and 
TAFE must be accountable. The NRS gives consistency and a framework for this. 

The NRS is important so that people can have a record of progress. 

We knew from the title that the NRS was meant to make it easier for skills to be 
transportable, and so that we could have a common language. 

General response to the NRS 

Most of those familiar with the NRS were positive about it overall. 

I really like it. It’s the best way we’ve ever had to describe language, literacy and numeracy skills. 

It can be very useful for determining language, literacy and numeracy needs. 

The best thing since sliced bread (but I don’t think vocational teachers get it). 

If I didn’t have to use it? Well I probably would now. I’m used to it and it’s a way of thinking 
about people’s skills and describing their achievement. 

However, there were occasionally other perspectives. 

To be honest, the NRS is a pain in the neck. 

The richness of teachers’ processes is not captured by systems such as the NRS. 

Although most interviewees were not of this mind, some believed that there were other practitioners 
who did not understand the NRS or appreciate its value. Some raised concerns about this 
perception, and about uptake and influence of the NRS. 

The NRS has raised awareness and established a common language and identified a potential 
for pathways and articulation where existing courses have been mapped. The question is, is 
this still happening or is it falling by the wayside? 
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If it had been able to produce what was intended, it would have been astonishing but the 
anaemic level of take-up justifies a revisit. 

Several academics commented that there was nothing equivalent elsewhere, and reported that 
overseas groups admired the multi-dimensionality and richness of the NRS. 

Whenever I talk about it at international conferences I get a lot of requests for the manual 
(but there aren’t any available!). 

You should hold onto the NRS. It is far ahead of what we have in the UK. We looked at it 
but felt our people were not ready for it. 

However, some other academics felt that the world had moved on. They suggested that the NRS 
be revisited in light of overseas developments, particularly in Canada, the United States and the 
United Kingdom. 

It is an incredibly rich instrument that was conceptually sound for its intended purpose. 

However, I do wonder if it is sound any longer. 

It should be seen as an artefact of its time. It reflects a conceptual understanding and 
environment of 20 [sic] years ago. 

It was ahead of its time, but perhaps not so any more. 

We need a less protective approach to evaluation. We need to be informed by those who are 
now ahead of us. The US and South Africa offer very good options. 

Theory base 

Amongst those interviewed, there was a strong consensus that the NRS was based on sound 
principles and theoretical constructs. It was seen by most to provide an effective framework and 
common language to analyse, discuss and report on language, literacy and numeracy. 

The theoretical underpinnings are sound, and very rich. It is an outstanding framework for 
understanding language. 

The construct makes sense. I use it every day. I think it. 

The basic premise is sound — it’s holistic with contextualised realistic tasks. 

It’s a solid useful framework. 

There were common threads running through interviewees’ views on the strengths of the 
conceptual base, and major issues associated with its use. 

Key strengths 

•y" acknowledgement of the complexity of language, literacy and numeracy, and the recognition of 
the importance of the range of factors that need to be taken into account in skills development 
and assessment 

•T recognition that language, literacy and numeracy are contextual 

•y" recognition that individuals are likely to function at a different level in each mode, and 
acknowledgment of the impact of various levels of support 

■v- sequencing of development across the levels, which was thought to ‘make sense’ and ‘reflect real 
life’ 

•y" inclusion of numeracy and the importance of a focus on numeracy not maths 

■v- recognition of learning strategies and the fact that they, and levels of support, have a bearing on 
language/literacy/ numeracy learning performance 
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•y" acknowledgement of a practitioner’s expert judgement (for example, the NRS design provided a 
useful level of detail without becoming prescriptive) 

•y" its applicability to different client groups, and potential to be used in conjunction with a range 
of curricula, training packages and assessment materials. 

Key issues 

Questions raised about aspects of the conceptual base included: 

Content and focus 

There is no evidence of critical literacy pedagogy in the manual. If there were, the document 
would be entirely different. 

Technology- related factors may not be reflected adequately in materials. 

Imaginative communication has been left out. For a significant number of people, ‘literacy is an 
invitation to move to the space of personal imagination’. 

The NRS needs to acknowledge multi-modality. For example, level 5 refers to skills for 
interacting with people from other language backgrounds yet, even people at level 1 are going 
to have to communicate with people from other languages. 

The emphasis on context and meaning making in literacy is not captured to the same extent 
in numeracy. 

The approach to learning strategies really needs to be revisited! 

Conceptual complication 

Most interviewees felt that the NRS was conceptually complicated. 

There’s some terrific stuff in the NRS but it tried to be too many things for too many people. 

It was produced by a committee! It needs to be simpler and much more useable. 

The progressive table is far too confusing. There are too many indicators of competence and 
the sequencing and numbering don’t align. The writers seem to have jumped around. 

It needs sophistication and elegance without complexity. 

It’s a matter of overly complicated theory vs necessary complexity. 

Although wanting some form of simplification, all interviewees rejected any attempt that might 
undermine the potential of the tool to recognise and manage the complexity of factors impacting on 
language, literacy and numeracy performance. As one person said, ‘You can’t dumb it down’. 

Rather it was a question of streamlining. Most interviewees agreed — often quite vehemently. 

Problems with the levels 

Although most people felt that the levels represented a natural sequence of development 
conceptually, there were issues raised. Many of these came from moderators of the Language, 
Literacy and Numeracy Programs and verifiers who have had the opportunity to test the instrument 
extensively across a range of users and contexts. Issues included: 

•y" too big a jump between levels 2 and 3 in both reading and writing 

■y- a confusing progression table (does not follow through from one level to the next) 

•y" not enough distinction between levels for oral communication 

■v" descriptive wording does not change enough between numeracy levels 1 to 3 to make clear 
distinctions 

•y" too many indicators of competence generally (for example, a need to capture learning strategies 
more clearly). 
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Observations included: 

A lot of teachers think there is almost room for another level between 2 and 3. 

When I first started using it, each level seemed to be saying the same thing! Now I can see fine 
distinctions. Moderation workshops and other things from the verifiers really helped with that. 

Another issue raised was that the NRS simply did not ‘go low enough’, particularly for people with 
some types of disability. 

The UK tools for literacy are very good and very clear — and they go below the NRS. 

Others felt that the NRS Level 1 was quite appropriate, and some suggested it should not be 
adjusted to try to cope with people with some forms of mental disability. Most practitioners 
reported that they had found ways of interpreting and managing NYA (not yet achieved), although 
one provider was critical that this should be the case. 

We’ve had to create a sheet called, ‘What is an NYA’ for our new trainers. 

A few practitioners were uncomfortable about labelling someone NYA for fear of undermining their 
self-esteem, although on reflection, they acknowledged this did not seem to be a problem in reality. 

Degree of detail and level of clarity 

Most interviewees remembered finding the lack of detail in the manual very challenging when they 
first started using the NRS, and this was borne out by those ‘first timers’ attending an introductory 
workshop. Two interviewees wanted to reduce ambiguity by developing more extensive checklists, 
but most respected the need to provide only a limited level of detail, and were comfortable with 
developing their understanding of how to interpret levels and indicators through involvement in 
moderation. 

It [the NRS] recognises professional expertise. I like that. 

It is not cut and dried and allows for teacher judgement. 

It is presented as being objective but it is really quite subjective. 

You develop a gut feel after a while. 

Some observed that incorporating more detail would turn the NRS into a kind of checklist and take 
the ‘thinking’ out of it. 

Teachers got drowned in the detail provided by some outcomes-based frameworks in schools. 

It didn’t work at all. 

Sharing examples and detail in moderation meetings is essential, but don’t put any more into 
the document! 

Interviewees felt that a distinction should be made between lack of prescriptive detail and lack of 
clarity. Some NRS statements were simply too vague to be useful, for example: 

Derives meaning from sustained oral text. What does that mean? No one has any idea! 

This aspect was seen as an issue that needed to be addressed. 

Numeracy 

A numeracy expert was pleased that numeracy had at least been included in the NRS, and agreed 
with the spirit in which it was addressed, but felt that it was not anywhere near as well developed as 
the literacy and language aspects. She suggested that this was largely due to the fact that significantly 
less time had been put into the development of the numeracy aspect, and was concerned that there 
had been no opportunity for those involved in the ‘pressure cooker’ activity to reflect on the basic 
construct. Nor had there been time to get practitioner input. 



22 



Reframe, rename, revitalise: Future directions for the language, literacy and numeracy National Reporting System 




Several other academics held similar views, as did some practitioners. 

It is harder to see the developmental sequence with numeracy compared with other modes. 

There are a lot of arguments at moderation! 

Despite these concerns, practitioners with adult numeracy training generally reported finding the 
numeracy component of the NRS an appropriate and useful tool. 

It’s a way of talking about the numeracy skills people have. It’s a reasonable representation of 
progression. I’d use it even if I didn’t have to — at least to talk to others who understood it. 

It’s a helpful framework for numeracy. 

In literacy there’s a big jump between levels 2 and 3 but numeracy is more logical. Someone at 
level 2 in numeracy can function in everyday life, but level 3 is more abstract. It’s a reasonable 
point of difference. 

Verifiers raised concerns that some practitioners had not grasped the difference between adult 
numeracy and school maths. They reported that some practitioners were still ‘setting sums’ with 
little real-life relevance. However, they recognised that using real contexts could create other 
problems, because they usually involved reading. Issues could arise when a person’s low literacy 
skills influenced their numeracy performance. One practitioner did not agree. 

It is easy to see how someone is functioning in numeracy, even if they don’t read and write well. 

Practitioners identified specific aspects of numeracy that could be reviewed, or presented more 
clearly. 

I wonder if there need to be four indicators at each level when literacy only has two. 

Four numeracy indicators is overkill. 

The wording isn’t always clear, and it can be hard to make distinctions between levels. The 
sheets put out by the verifiers often explain things much better. 

It should be in plain English. I mean what about changing things like, The exchanging of goods 
for services ? 

Appropriateness for different groups 

A number of practitioners could speak from first-hand experience of using the NRS in diverse 
settings with diverse groups for diverse purposes. They saw this as an indication that the NRS had 
achieved its original intention of being useful in a range of contexts with people of different 
backgrounds. 

I’ve worked all over and used the NRS everywhere in different ways. The workplace is different 
from TAFE [which] is different to community literacy programs. You don’t need all of the NRS 
for every situation, but you can tailor it to suit. I take from it what I need. For example, the 
levels are really useful in the workplace but we don’t use the aspects at all. They aren’t relevant. 

The flexibility to contextualise is very important when working with Aboriginal people. 

People with a disability may have low oral competence. The NRS makes it possible to 
demonstrate that they may well be much more competent in other areas — but you do need 
expertise with disabled people and with the NRS to show this! 

Since its inception, there have been disagreements about the ability of the NRS to cater for both 
native and non-native speakers (including users of AUSLAN [Australian Sign Language]). Some 
interviewees felt the NRS was not entirely suitable, but others found it quite applicable. 

It is difficult to have one system for reporting on literacy and ESL [English as a second 
language] issues — they have different scripts, and different ways of organising that may not be 
immediately obvious. 

It works really well with migrants, especially those with high skills levels in their own language. 
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I am TESOL-trained and find the NRS works well for ESL and native speakers in the 
workplace. There’s no need to use different approaches. 

In numeracy one word can make a difference to how you interpret a problem. This can make 
a difference to an ESL person’s assessment as they can have an interpretive problem, but 
actually be good at numeracy. You have to watch for this. 

Some people say it lacks the subtleties of the ELS documents, but I think it’s mostly that 
people are wedded to the frameworks they know! The upper levels of the ISLPR 
[International Second Language Proficiency Ratings] have no subtleties at all! 

Several practitioners who worked with Indigenous learners from non-English speaking backgrounds 
reported finding the NRS culturally appropriate overall because it allowed them to develop 
curriculum and assessment materials relevant to the learner’s context and aligned to NRS levels. 
Although there were cultural differences, such as the emphasis placed on oral language in Aboriginal 
culture, they reported finding the NRS framework and reporting rules flexible enough to allow 
them to interpret the indicators appropriately for their clients. 

The name 

There were mixed responses to the name, and the possibility of a change. 

The name is misleading. It was intended as a reporting mechanism — it is not really a ‘system’ 
at all. A system would collect a range of data re outcomes of learning. The NRS would be one 
component only. 

It doesn’t really tell you what it is does it? But I guess we are used to it. 



The NRS as a reporting tool 

When used as a formal reporting tool, there are two components to be considered — the instrument 
itself, and the structures, processes and rules designed to ensure that ‘reporting’ produces the 
information required by identified stakeholders in a way deemed to be valid, consistent and fair. 

This section considers interviewees’ views on the effectiveness of the instrument for evaluating 
language, literacy and numeracy competence, and of the appropriateness of the formal processes 
associated with the LLNP and WELL. The focus was kept on the capacity of the NRS to fulfil 
general reporting requirements. It was not within the scope of this study to consider issues 
associated with the detailed logistics of WELL and LLNP, so issues raised in this regard were 
recorded but not explored in depth. 

Where is the NRS used for reporting? 

The NRS is currently mandated for use in reporting on two programs funded by the 
Commonwealth Department of Education, Science and Training — the Language, Literacy and 
Numeracy Program (LLNP) and the Workplace English Language and Literacy (WELL) program. 
Although both are concerned with adult language, literacy and numeracy, the programs have quite 
distinct purposes and requirements, and the NRS is used differently in each. 

In the LLNP, the NRS is used to determine the eligibility of an individual for funding to cover the 
cost of participation in general adult literacy and numeracy programs, and for setting benchmarks 
against which to measure a learner’s progress. The expectation is that a learner will make measurable 
progress over time. The Department of Education, Science and Training uses the information 
collected to report on job seekers’ language, literacy and numeracy proficiency gains at an 
aggregated level, and to evaluate the cost-effectiveness of the LLNP. 

Providers are required to conduct pre- and post-training assessments. They must maintain detailed 
documentation, including assessment material used and explanations of the thinking behind the 
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assignment of an NRS level. These are submitted for validation, and receive extensive feedback 
from an experienced team of validators. Practitioners may also attend yearly moderation sessions 
designed to promote consistent interpretation of levels. Both validation and moderation are 
conducted by highly skilled professionals, appointed through a tendering process. 

The WELL program aims to provide workers with the language, literacy and numeracy skills they 
need to meet the requirements of current and ongoing employment and training. NRS levels are 
only used formally at the end of the program. Reporting against the NRS is seen by the funding 
body as a way of reinforcing the workplace focus on language and literacy as opposed to general 
training. It also provides a profile of the language and literacy levels of the group involved. This 
helps the Department of Education, Science and Training to determine whether the funding has 
been used for the target group, predominantly those with an NRS level of 3 in at least one mode. 
Level 3 has generally been recognised as a minimum requirement for effective functioning in most 
workplaces, and an assumption behind the program has been that individuals with NRS levels 
below this would be more suited to a different approach (although interviewees reported an increase 
in participants operating at lower levels in the mode(s) or aspect(s) on which a particular workplace 
program focused). 

There is no formal moderation/validation process as in the LLNP. However, site data are entered 
electronically by providers and checked for anomalies by state and Commonwealth Department of 
Education, Science and Training staff. 

Table 1 provides a comparison of the role of the NRS in these two Commonwealth-funded 
programs, while appendix 3 contains two contrasting practitioner perspectives on the NRS. 



Table 1 : Use of the NRS within LLNP and WELL 





LLNP 


WELL 


Program focus 


To assist those with low levels of language, 
literacy and/or numeracy 


To improve workplace language, literacy and 
numeracy 


Target group 


Clients at ‘Not yet achieved’ or LI and 2 


Target group NRS L3 




May not be in workplace but intention to 
improve potential to gain employment as well 
as to operate effectively in the community 


Workplace-based 


NRS purposes 


Pre-training: assessment to determine 
eligibility for program and allocation of hours 

Post-training: to indicate progress 


Post-program to provide a snapshot of 
participants, and demonstrate that WELL is 
reaching main target group 


Focus 


Individual 


Snapshot of all participants in program 
No individual progress mapped 


Assessment 


Across all 13 NRS indicators 


Only indicators and aspects relevant to 
training focus 


NRS determination 


Area of lowest performance reported pre- 
training 


Area of highest performance reported post- 
program 


Funding body 
expectations 


Participants will move up a level 


Participants may stay within level but will 
develop specific skills within current context 
and potential to transfer these to other 
contexts 


Validation 


Formal documentation submitted for validation; 
written feedback 


Electronic reports of highest level achieved 




Interactive moderation process — provider 
participation once a year 




NRS professional 
development 


Introductory courses held regularly in some 
states (e.g. SA funded by state bodies) 

Internal mentoring 
Moderation meetings 


Responsibility of provider 
Internal mentoring 

Industry Training Councils and professional 
bodies may be included in professional 
development program 
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Perceptions of the effectiveness of the NRS for reporting 

Most interviewees stated they accepted the need to show progress, set standards and be accountable 
for funds. 

You need a reporting tool. Otherwise you can’t have standards. You can’t benchmark. 

It shows progress. How else can you really do that? 

The context within which the NRS was used for reporting may be a factor influencing perceptions 
of the NRS as a whole. Although some had complaints about the logistics and time involved in 
reporting to funding bodies, most LLNP practitioners interviewed felt that the NRS had improved 
their ability to work with learners in a range of capacities. However, WELL practitioners tended to 
be more critical, with some questioning the effort that went in to learning about the NRS for so 
little valued return for teachers or learners. 

The NRS and the Language, Literacy and Numeracy Program 

Experienced LLNP practitioners were generally comfortable with using the NRS for reporting 
purposes. They found the NRS was helpful in determining where a person was and in tracking 
progress, and useful for developing assessment tasks. 

They also took it very seriously. 

[In the LLNP], the NRS is critical. If you got it wrong you would lose funding and set people 
up for failure. You’ve got to get your head round it to use it effectively. It’s almost more 
important than the curriculum. 

No one who actually used the NRS reported feeling that it had a negative impact on the learner. 
On the contrary, they generally believed it was an asset. 

My personal philosophy is that everything must help the learner to be successful. The NRS 
supports this. 

It is helpful and effective. It doesn’t undermine relationship building, teaching or learning. 

Consistency and validity 

All LLNP practitioners interviewed felt they were able to conduct an NRS pre-training assessment 
appropriately and use it to show progress over time. Some said they found preparing a report very 
productive because it promoted reflection on student progress, and helped them weigh up where to 
go next. 

It’s part of the work, not an added burden. 

Those who had never used the NRS in the field were more inclined to doubt its efficacy and 
question its validity. Several non-users mentioned hearing of practitioners who conducted pre- 
assessments by getting clients to begin at level 1 and work their way up until they could not cope 
any more. 

One interviewee who had observed NRS pre-training assessment processes as part of research into 
the LLNP’s predecessor, the Literacy and Numeracy Training Program felt that the needs of the 
learner were forgotten in the push to provide reporting data. In studying six practitioners she 
observed: 

•T pre-training assessment interviews that went as long as two hours, with a ‘bizarre’ switch mid- 
way, from ‘getting to know you’ to ‘now do this test’ 

•y" assessors overwhelmed by the complexity of the NRS requirements trying to work their way 
through the document as they interviewed 

•y" an (unsustainable) notion that the NRS was in some way ‘objective and scientific’. 
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She believed that the needs of the learner were being ‘sacrificed’ to the need to please the verifiers 
and was highly critical of a process she believed was leading to stress, skewed results and impaired 
relationships. 

In contrast, LLNP practitioners interviewed made a point of stressing the lengths to which they 
went to minimise negative impact on people’s self-esteem; several referred to the videos available to 
model appropriate approaches and the focus on this issue in training and moderation sessions. 

Critics also questioned the validity of a process in which the provider had a vested interest in 
identifying someone as a level 1 or 2 in order to qualify for funding, and who was then under 
pressure to demonstrate a result. 

I’d question how it is really being used. Anecdotal evidence suggests it is easy to report so you 
can get the funding for the LLNP. 

It was not possible in this study to ascertain the validity of the differing perspectives. However, the 
fact that there is a degree of cynicism regarding the validity of the process amongst some influential 
non-users is of potential concern. 

LLNP practitioner issues 

Practitioners identified a range of issues associated with the structures and processes relating to rules 
of reporting. There was strong consensus on the following issues: 

One-number reporting and ‘rounding down’ 

These issues were the hot topics in a consideration of reporting rules. 

A number of interviewees expressed concern about the need to reduce a learner’s varied levels of 
performance across all modes to a single level equal to their lowest level of competence. Although 
this practice of ‘rounding down’ is formally endorsed within the LLNP (Fitzpatrick, Wignall & 
McKenna 1996, p.13), a number of people spoke about the practice as if it were somehow 
‘cheating’. During the introductory NRS workshop, it was a topic of debate between the presenter 
and a practitioner working with English as a second language clients outside the LLNP, who felt 
that rounding up was more applicable to her clients’ actual skills. Those who were not part of the 
LLNP saw rounding down as an example of questionable practice. They suggested that it left the 
system open to manipulation by providers who would benefit from an allocation of more hours. 
Interestingly, a private provider reported that her organisation had not been using the practice 
because it did not align with the tailored approach they used, but that the decision had reluctantly 
been made to round down in order to ensure that each client was eligible for the appropriate 
number of hours. 

Several interviewees were uncomfortable about one-number reporting because they saw it as a 
reductionist approach which labelled a person with the lowest level when this did not reflect their 
actual range of performance. However, on reflection, they decided that their initial reaction was 
more an indication of the importance they placed on building learner self-esteem, and their feeling 
that the client would feel bad about being a ‘level T. They acknowledged that, in practice, this did 
not seem to happen, and that, in fact, it was more likely to work in the learner’s favour, because it 
meant the learner was eligible for the maximum hours of training. 

Reporting across all indicators 

Assessors are required to assess and report on the 1 3 NRS indicators of competence across all modes 
in a level. This is reinforced by the software design, which will not accept a report until all fields are 
filled, and by the verification process, where verifiers follow up on anyone who has not provided 
sufficient background information to support an entry. 
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Several practitioners, and a verifier, suggested that there were instances when it was not appropriate, 
or possible, to assess all areas, or provide the level of detail required. In these cases, they felt the 
requirement undermined the basic principles of the NRS concerning fairness, functionality in 
practice, and the promotion of good educational practice. 

Providing evidence 

All those interviewed were very positive about the interactive nature of the LLNP moderation and 
validation processes. Despite the time involved in preparing documentation, most valued the 
opportunity to reflect in a structured way on each learner’s progress, and felt that it contributed to 
their effectiveness as teachers. However, there were specific problems in supporting their 
assessments, particularly in numeracy. 

It is very difficult to demonstrate for moderators that a student can estimate and self-correct. 

In relation to ‘Reflecting on the answer’, this indicator is probably very sensible, but it is very 
hard to provide evidence for this. 

Plateauing 

Another concern was the necessity to show that a learner has progressed to a higher NRS level 
within a mode as a result of training. Several practitioners reported that some learners reach a point 
where they do not seem to be able to progress any further upwards. However, they are usually 
broadening their skills by learning ‘sideways’ across competencies within the level. Practitioners 
were comfortable that this was appropriate for the individuals involved, and was not a negative 
reflection on their teaching, but felt it was unfair that attaining further competencies within a level 
was not captured and valued by the LLNP reporting system. 

Non-completion 

Several interviewees mentioned that some of their clients left the LLNP before a final assessment 
could be conducted. However, most were not overly concerned from the learner perspective because 
they felt their clients knew they had made gains, and because many left to take up newly found 
employment. However, there were concerns where this prevented the provider from documenting 
the learner’s success, or where it could be used as an indictment of the program. 

Time 

Several practitioners stressed the importance of preparing reports as soon as possible after an 
assessment. 

I always try to write it up immediately. The more time that passes the harder it is to stay 
focused, especially for learning strategies and oral communication. I’m OK, but I’m aware of 
other trainers who simply don’t have the time and can’t make the time. 

The time taken to enter results electronically was often raised as an issue, but most said that this 
became less of a problem as they became more familiar with the instrument and with the software. 
However, no matter how streamlined the process, they pointed out that there would always be a 
time factor involved. There were indications that some organisations were better than others at 
factoring this into workloads, or at providing specific assistance to make the task less onerous. For 
others, it was not so easy. 

I’ve been a bit too proactive and managed to double our client load. The trouble with this is 
that the administrative allowance hasn’t changed! 

The NRS and the Workplace English Language and Literacy program 

WELL practitioners interviewed were highly supportive of the WELL program itself, but tended to 
be less positive about the use of the NRS within it than were LLNP practitioners. 
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Few of those in WELL were able to identify a useful purpose behind the application of the NRS 
because it was only used to show the level of each member of the group at the end of the course, not 
as a way of tracking progress, and no individual records were submitted. 

Some assumed that the funding body expected progress to occur, yet did not ask for NRS evidence. 
Several were critical of this. 

WELL needs to pull itself up [re the NRS] . The NRS is not being utilised as well as it could 
be. You actually need to assess each person involved so you can show progress, 
but there is no requirement to do this, even though it’s easy and quick to do in an industry 
context. What’s the point? 

However, most did not actually want to go to the trouble of formally assessing and reporting on 
individuals, believing this would be unnecessarily time-consuming and that it would undermine the 
positive interaction of WELL programs. Some practitioners said they had difficulty in teasing out 
specific competencies for the purpose of reporting, given that language, literacy and numeracy are 
being used in context, and courses tend to be of short duration. 

Two interviewees described the use of the NRS in WELL as ‘overkill’, questioning the amount of 
effort that went in to learning about the NRS when it was ‘not really used’. They each suggested 
that this was an important issue that needed to be addressed for several reasons: 

•y" decreasing levels of background knowledge and experience of practitioners, and poor availability 
of professional development generally 

•y" the high turnover rate, meaning that providers were forever needing to train newcomers in the 
use of the NRS, using funds that could have gone into the program itself, or into further 
professional development 

•y" the high proportion of sessional trainers employed in WELL programs who were unlikely to 
receive enough professional development on the NRS. It was suggested that some people using 
the NRS had received no formal instruction at all. 

However, there were several practitioners who questioned the reporting requirements but still found 
the instrument of value. One, working with people from English as a second language (ESL) 
backgrounds in the textile industry, felt that: 

The NRS acknowledges that you need to cover a number of aspects and competencies. It can 
put pressure on people to go broader and wider than they might otherwise. The descriptors 
are really good when you are trying to think about how you might approach developing 
materials and assessing. 

Reporting outside Commonwealth programs 

It does not appear that any states or territories have formally adopted the NRS as a reporting tool 
within their own programs. Several state education and training personnel interviewed had 
considered the possibility, but decided against it. 

If we want to report the levels of literacy of those who undertake VET courses, we look at the 
certificate levels, which have been mapped against the NRS. An NRS report would duplicate 
what has already been done. 

We’ve already got enough reporting requirements to ANTA and others. You can’t ask people 
to do too much. 

There were also concerns that information from an NRS assessment might be misused as a 
screening tool if, for example, prescribed NRS levels were required within the New Apprenticeships 
and traineeships scheme. 
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Although it is not required within ACE programs, several of those practitioners who attended a 
recent introductory workshop had enrolled because they wanted to be ready if the NRS was made 
mandatory. (A representative of the state funding body involved said that there were no plans to 
introduce the NRS as a requirement. However, the conversation has led to a renewed interest in 
utilising the NRS as a professional development tool and as a consistent framework for describing 
language, literacy and numeracy across the state ACE sector.) 

Several interviewees also raised the potential of using the NRS to report on language, literacy and 
numeracy levels of school leavers. The NRS could act as a bridge between the schools sector, ACE 
and VET. A model incorporating this approach is currently under discussion in one state. 



Non-reporting applications 

Although developed primarily as a reporting tool, it had always been anticipated that the NRS 
would be useful for a range of purposes. However, some academics who had been involved in the 
conceptualisation of the NRS questioned the appropriateness of using it for non-reporting purposes. 

It was designed for a purpose and you can’t fiddle with it. 

We need to review the NRS and its intentions and explore whether it’s the kind of instrument 
to deliver in other areas. 

You have to ask: What do you want it to do? Was it designed for this? 

However, other interviewees were already using the NRS — or at least its concepts — to inform a 
variety of adult literacy and numeracy activities. 

The NRS as a tool for reflective practice 

There was strong support for the NRS as a tool that facilitated reflective practice. Some 
practitioners wondered what they had ever done without it. Many talked about the importance of 
having a common language. The role the NRS now plays in this regard was evident at a recent 
Australian Council for Adult Literacy conference, where participants often referred to NRS levels in 
workshop conversation as a shorthand for a parcel of descriptors of performance. 

As one practitioner educator observed: 

The NRS gives the teachers something with which to analyse what students are doing. Before 
the NRS they worked more haphazardly in assessing literacy and numeracy levels, sometimes 
forgetting things. The NRS provides them with a ‘language’ with which to convey ‘where a 
person is at’. 

However, another experienced teacher-trainer observed that, although the NRS certainly helped 
teachers to gain insights, it was still a complicated mechanism. 

If using it in training, you must look at the NRS every time and know how to do it to 
genuinely change pedagogy. This is a Big Ask! 

She felt the key was to have a clear purpose for using it. It was not a tool you could somehow learn 
and use in isolation. This message about purpose came through from many practitioners. 

The NRS as a tool to develop or evaluate curriculum 

The NRS pilot evaluation (Cumming 1997, p.8) reported: 

An unexpected outcome of the project was the extent to which participants identified a role 
for the NRS in curriculum development and review. This occurred both for general basic 
education and for language, literacy and numeracy embedded in vocational education. 
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Eight years on, several interviewees who had been involved with the NRS since its inception 
identified this role as one of its most important. 

It’s biggest value is as a curriculum framework. It helps you to map what you are doing. Once 
you are familiar with it, you don’t have to go back all the time. 

The NRS works for mapping curriculum. It helps that the state curricula are all from the same 
family, informed by the same theories and influenced by cross-fertilisation. 

It is not set up as a curriculum framework but it would be more useful if it were. 

There was occasional confusion about the use of the NRS as a curriculum framework or as a 
curriculum , with several practitioners expressing concern that any move to make the NRS a national 
requirement would actually mean that it would become the standard curriculum for basic adult 
literacy and numeracy. They did not like the idea of giving up the state curriculum they were 
comfortable with. 

Mapping to the NRS 

The general adult language, literacy and numeracy curricula in use around Australia have all been 
mapped to the NRS. Practitioners interviewed for this survey were all aware that this mapping had 
occurred in relation to the curriculum they used, although there were mixed responses to the 
usefulness of the process. 

Our curriculum preceded the NRS. It was easy to map to it, but it was not informed by it. 

The GCEA [General Certificate in Adult Education] numeracy doesn’t map together with the 
NRS very well, but in practice we get round it by using different tasks for the NRS assessment. 

One practitioner who did not actually use the NRS for reporting felt that mapping existing 
curricula to it gave ‘a sense of security’. Another who had been involved in a mapping process 
conducted soon after the publication of the NRS, remembered: 

It created a lot of dissent and was not done very well because we were not really familiar with 
the NRS. Lucky it’s up for review soon! 

However, she acknowledged that the discussion at the time had been valuable, even though the 
outcome was less than perfect. 

A different view was put by an academic who suggested: 

We should question the relationship between the NRS and the curriculum — there does not 
seem to be much of a relationship . . . 

And another was concerned that the NRS could be used to justify anything. 

Most people use their own curriculum and map it to the NRS if they have to. It doesn’t 
matter if the curriculum is reactionary. It will still map to the NRS because there is not 
enough detail to drive good practice. 

Developing new curriculum 

There were concerns raised that some practitioners might be treating the NRS as a sort of 
curriculum in itself, but most interviewees found it hard to imagine how this could happen. 

Several reported finding the NRS helpful when developing new curriculum. For example, a 
Queensland trainer has used the NRS as a framework to develop an accredited course in the retail 
sector which helps workplace trainers explicitly address the language, literacy and numeracy 
requirements of certificates I, II and III. 
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NRS as a tool to identify and benchmark communication demands of 
training packages 

The integration of language, literacy and numeracy into training packages has been ANTA policy 
since 1995, and is captured by the phrase, ‘built in not bolted on'. Developers of training packages 
are required to take account of the NRS as part of the development process. There is no required 
format, but a model based on the NRS is provided for use by standards developers and training 
package reviewers (ANTA 2001a, p.8). 

Interviewees generally supported this application of the NRS, but identified problems with it in 
practice — along with more general issues associated with language, literacy and numeracy in 
training packages. 

Several interviewees felt that some package designers had not really embraced the NRS, and were 
only going through the motions because they were paid to. Their lack of expertise in the NRS could 
lead to misinterpretation of NRS levels. 

In discussions with interviewees for this study, some suggested that the real problem with the 
application of the NRS was not so much with the designers, as with workplace trainers who had 
little or no literacy and numeracy knowledge. It was not just that the NRS was not understood or 
appreciated in the workplace, it was that issues related to language, literacy and numeracy were not 
understood. This feedback aligned with the findings of various research projects that the theory of 
integrated language, literacy and numeracy has not necessarily been translating into practice (Haines 
& Bockmore-Brand 2000; McGuirk 2000; Wyse & Brewer 2001). 

Although supportive of the notion of embedding, most interviewees were concerned that this 
actually made it harder for workplace trainers to focus on literacy and numeracy, and cited many 
occurrences in their experience in the workplace. 

The problem with embedding was powerfully demonstrated during an Australian Council for Adult 
Literacy national conference workshop (Yeomans & Krusche 2004), when some 40 experienced 
practitioners attempted to identify the literacy and numeracy competencies required in two 
different training packages. Although possible, it was a challenging task, made even more difficult 
because the NRS was not used as a mechanism to identify and evaluate the expectations built into 
the two packages. Besides demonstrating the inherent difficulties facing trained practitioners (and 
therefore raising questions about the potential for workplace trainers without this background to 
tackle this effectively), it was also interesting that the workshop presenters had not used the NRS as 
part of their Masters’ research study. However, several participants asked about the NRS levels, and 
it appears that, in most small groups, there were practitioners who used the NRS indicators as a way 
of making sense of the tasks and of establishing a common base for discussion. 

The NRS and the Australian Qualifications Framework 

Several interviewees drew attention to the fact that there is no simple correspondence between NRS 
levels and Australian Qualifications Framework (AQF) levels generally, or between the language, 
literacy and numeracy requirements of certificates at the same level in different industries when 
measured against the NRS. They did not see this as a fault with the NRS, but as a reflection of the 
reality that it is crucial that the language, literacy and numeracy demands of any work context be 
considered individually. However, some suggested that this basic message was not necessarily getting 
through to trainers, who lacked the background knowledge to grasp what it meant. Again, they 
argued that it was not the fault of the NRS. It was the complexity of language, literacy and numeracy 
in context that caused the problems and highlighted the need for training for non-practitioners. 
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Some interviewees cited examples of a misalignment between the literacy and numeracy requirements 
within a workplace, entry requirements for training, and literacy and numeracy levels required to 
achieve competence in a course. The NRS can been used to highlight such issues, but its impact is 
limited, due to a general lack of understanding of the instrument and its underpinning concepts. 

The NRS and workplace trainers 

One suggestion posed as part of this brief was that the NRS be simplified to make it more accessible 
to workplace trainers without specialist literacy and numeracy training. However, no interviewees 
supported this notion if it meant that the NRS lost its bite. Nor were most practitioners comfortable 
with the images conjured up by a simplified NRS — which they tended to associate with a checklist 
that could be used to misjudge and misreport. 

However, there were also those who could imagine a simplified version with positive applications 
because they had already developed and used them to raise awareness of language, literacy and 
numeracy issues amongst non-specialist trainers and industry members. Several of these simplified 
outlines had been produced, although it was important to note that they were not presented as 
‘stand alone’ documents, but were used within workshops and seminars. It was reported that non- 
specialists had no problem grasping the basic concepts when presented in this context. The issues 
only arose if they tried to move to the full NRS and saw the manual! 

The NRS and assessment 

One practitioner who had developed a range of context-specific assessment materials mapped to the 
NRS stated: 

The professional development sessions pushed the fact that the NRS was only a reporting 
tool, but it has always been used more broadly than intended. For example, you weren’t 
supposed to base assessment on it but we always have! 

This was an interesting perception, because using the NRS as an assessment framework was always 
part of the NRS scope and intention. However, it reflects some confusion over use of language. 

Often when people reported ‘using the NRS for assessment’, closer questioning revealed that they 
actually meant they had designed assessment tasks relevant to the learner’s context which had 
helped to identify appropriate levels. Many of those interviewed worked in organisations which had 
developed ‘assessment kits’ that could be used as part of pre- and post-assessments. They reported 
that these offered enough choices, and were appropriately contextualised to meet the needs of 
different individuals. 

For example, one large government organisation had used WELL funding to develop a tool based 
on the NRS to help identify where employees would need language, literacy and numeracy support. 
Modified many times over the years, it has become an integral part of the program to improve 
language, literacy and numeracy in its workforce, and provides a consistent way of talking about 
people’s proficiencies and progress across an organisation with over 14 000 employees. 

Within this organisation, job and person specifications are analysed by a literacy and numeracy 
specialist against the NRS, and used to inform recruitment — but not to screen people out of a job. 
The NRS simply helps identify when an applicant who meets other requirements will benefit from 
additional support in literacy or numeracy in order to do the job effectively. Although this is an 
application that was identified by NRS developers as problematic, it is reported to be working well 
for individual staff, while raising awareness amongst management about language, literacy and 
numeracy and demonstrating that a focus on adult literacy and numeracy can make a difference for 
both the individual and the organisation. 
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Requirements for effective use of the NRS 

There was general agreement that the current NRS was not easy to learn or to apply initially. Even 
those who had been using it for some time wanted opportunities to compare their interpretations 
and share ideas about assessment materials. Interviewees identified four essential components of an 
overall ‘package’ to support NRS use as a formal reporting tool. These were: 

V a level of background knowledge of language, literacy and numeracy sufficient to be able to 
appreciate the conceptual aspects of the NRS and interpret them appropriately 

•T regular opportunities for interactive professional development activities 

■y- accessible, relevant support materials 

-y" time. 

If any one of these were missing, many felt it would be hard to sustain the NRS (current or revised) 
in the future. 

Interestingly, although all interviewees raised issues about the ‘daunting’ nature of the original 
manual, and accessibility of later support materials, they were more concerned about the lack of 
ongoing NRS professional development. They also identified looming problems associated with an 
ageing specialist workforce, and a trend towards less qualified practitioners with limited access to 
either NRS professional development or workplace mentors. Without a strong training support 
program, this had the potential to undermine the future effectiveness of the NRS as a reporting tool. 

Background knowledge 

The NRS was originally developed for use by literacy and numeracy professionals. This approach 
was endorsed by those interviewed. There was strong resistance from many to any move to allow 
non-professionals to use the NRS for assessment and reporting purposes, or to simplify the 
instrument in an attempt to make this feasible. 

It could be a dangerous tool in the wrong hands. It must be used in a safe professional 
environment. 

Arguments about qualification levels required for the NRS reflected broader discussions currently 
occurring in the adult literacy and numeracy field about qualifications generally, and in relation to 
the literacy and numeracy components of the Training and Assessment (TAA) Training Package. 

Interviewees suggested that the NRS required a sophisticated conceptual understanding. Most 
argued that users needed a background in adult literacy and numeracy, preferably with post- 
graduate qualifications, and probably a teaching degree. Although some interviewees were hesitant 
to be too prescriptive, most said: ‘More than a certificate IV!’ At the same time, the small number of 
interviewees who did not have post-graduate qualifications raised the issue themselves in light of 
ongoing debates about qualifications, and expressed annoyance at any suggestion that they were not 
effective in their roles. They believed they were quite capable of interpreting the NRS appropriately, 
and were doing so on a daily basis. 

A few interviewees with qualifications felt there was a need to be pragmatic and recognise the 
trends — an ageing workforce, casualisation, and few young people entering a profession that offered 
little in the way of a career path. With very few people actually willing to pay for post-graduate 
qualifications (at one university numbers had fallen from 60 per year to 6), they might soon become 
a thing of the past. This group argued that a Certificate IV in Literacy might become one of the 
only forms of specialist training available. While no one thought that the current literacy and 
numeracy component of a Certificate IV in Workplace Training and Assessment was sufficient to 
enable a person to assess and report against the NRS, they were hopeful that the new Training and 
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Assessment Training Package would provide a solid grounding so that a trainer would have a better 
idea of when to get a specialist in to help. 

Professional development 

All interviewees saw interactive professional development as one of the most crucial aspects 
associated with the effective use of the NRS, and most felt it had not been adequately addressed. 

The key point made was that the NRS is conceptually challenging and not easily learnt from a book 
or website. 

The NRS doesn’t suddenly become knowledge for new people because they attend a half-day 
workshop. It will have zero effectiveness if it is used independently of regular professional 
development. 

Interviewees were universally critical of the fact that there was no organised approach to NRS 
professional development, and concerned about the limited availability of any kind of professional 
development in the language, literacy and numeracy field. 

The professional development aspects need updating. The NRS can’t expect to stay live for 10 
years without it! 

It was argued that professional development should go beyond a familiarisation with the instrument 
and an introduction to the requirements of WELL or LLNP. Increasing proficiency in its use only 
came with time, experience and regular opportunities to explore, reflect and share ideas. Therefore, 
interviewees suggested that several different types of professional development should support the 
NRS in its current form and roles or in any future guise: 

•T an introduction to the NRS from a conceptual point of view 

-y" an introduction to the rules of formal reporting within WELL or LLNP 

•y- regular updates and opportunities to discuss NRS application with other practitioners. 

Learning the NRS 

To further explore the best ways of introducing the NRS to new users, practitioners were asked 
what supports had been most helpful to them in developing their understanding. 

Learning from others 

Some people had no formal introduction, but learnt from others within the organisation. 

I would have found it much harder, but the program coordinator had prepared an assessment 
kit and selected key aspects that made it much easier to get into the NRS. 

The mentoring approach worked well if these instructors were experts, and had the time. However, 
this was not always the case. 

It’s very complicated for coordinators and lecturers to explain how the NRS works to hourly- 
paids who are only there for the time they are actually working. 

Given the high turnover in some places, there were reports that the chain of experienced NRS 
practitioners helping newcomers was being broken. When this happened, information and 
observations about how to use the NRS were being passed on by an inexperienced practitioner. 

Over several iterations, key messages were being watered down. It was also possible to reach a point 
where no one actually had any knowledge any more. Interviewees from several states reported that 
representatives from some organisations had been sent to a recent NRS introductory workshop in 
another state for just that reason — because there was no longer anyone with any NRS background 
available who could show them what to do. 
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Attending provider professional development programs 

It is Department of Education, Science and Training policy that the provider be responsible for 
ensuring that WELL practitioners are trained in the use of the NRS as a reporting tool, while the 
LLNP has a built-in moderation and verification component that acts as an effective professional 
development mechanism for those able to attend. Some providers (such as ACE NSW, Mission 
Australia and some state TAFE systems) have strong professional development programs for those 
working in adult literacy and numeracy, and regularly cover the NRS for those required to use it 
within Commonwealth-funded programs. 

However, most practitioners interviewed, from whatever background, reported that professional 
development generally was a ‘thing of the past’. It had become the individual’s responsibility and 
was unlikely to be paid for, or even supported with time release, by their organisation. In these 
circumstances, NRS professional development was hard to find, and most people relied on 
Language, Literacy and Numeracy Program moderation meetings as a way of developing their 
knowledge and skills regarding its use. However, these were not available to non-LLNP personnel, 
and as all sessions are oversubscribed, this avenue is not currently open to everyone involved in the 
LLNP either. 

Although representatives of funding bodies felt that it was quite acceptable for providers to include 
the cost of professional development in tenders, practitioners and some providers made the 
assumption that this was not appropriate. There was also evidence of a widespread belief across 
most practitioners that the funding bodies mandating the use of the NRS should be directly 
responsible for the professional development associated with it. 

Participating in LLNP moderation 

Moderators interviewed for this project reported designing the workshops to encourage reflection 
and extend understanding of literacy and numeracy through the inclusion of guest speakers, and 
reviews of resources. Interviewee feedback suggests that this approach has been highly successful. 
Those interviewed identified moderation meetings as the major source of learning for LLNP 
practitioners, and spoke highly of the experience. All found it extremely helpful, not only in terms 
of NRS use as a reporting tool, but as a way of reflecting on their practice and sharing ideas with 
peers. Although they were keen to ensure they reported accurately, they placed a high value on the 
opportunity for structured professional interaction offered by moderation meetings. 

However, lack of places meant that most organisations could only send one representative a year. 
There were also issues raised about access for practitioners working in regional and remote areas. 

Receiving feedback as part of LLNP validation 

Practitioners reported being initially concerned about ‘being marked’, but then found that the 
validation process was a positive experience that tended to validate their approach as well as their 
assessments while also extending their thinking and practice. 

However, one validator was not so sure. She reported that she had taken on the role because she saw 
it as an opportunity to provide professional development, but had found she spent far too much time 
‘ticking boxes’ and checking whether people had included enough information. She was also critical 
that the ‘professional development’ aspect tended to focus on compliance and rules of reporting 
rather than on concepts and strategies that could lead to good teaching practice. Nor did she believe 
there was much opportunity to develop any continuity. Given the high turnover of trainers, and the 
time between meetings, she found that each meeting had to ‘go back to the beginning’. 

State-funded programs 

Some state education and training agencies provide funding for introductory and advanced NRS 
courses, which may be offered several times a year. However, this practice does not necessarily occur 
in all states. 
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Through professional and industry bodies 

An Industry Training Council representative described how his body provided literacy and 
numeracy training because it was felt someone had to ‘fill the gap’. The program was extremely well 
attended and attracted practitioners who did not actually work in the industry sector. NRS training 
was offered about once a year as part of a general program. 

Resources 

The NRS document 

Although one person described the original NRS manual as ‘ingenious’ and another as ‘an 
architectural and engineering marvel’, interviewees were almost unanimous in their criticism of the 
original NRS document, suggesting that its complexity created a psychological hurdle — it simply 
looked too hard. 

As soon as you open it, it looks daunting. 

It’s daunting. 

I can remember being daunted by it! 

It’s really hard to juggle all the different aspects at once. 

Oh that book! 

Its tone and style also worked against it. 

It’s verbose and scary, yet as a literacy document it should be easy to read. 

Accessibility is a huge weakness. 

Those who had overcome initial barriers and become experienced users complained that the original 
NRS manual was very difficult to get hold of, and asked that more be made available. 

Support materials 

A wealth of materials has been developed since the publication of the original NRS manual. Most 
are available on the web, but not all in the same place. (Initial searches for this project took a long 
time and were not straightforward.) Some long-time NRS users were not even aware of much that 
had been published post-1996. However, those who were, spoke highly of the work that had been 
undertaken to make the NRS more accessible, and wondered why it had never been incorporated 
into the original package. 

Although interviewees from Commonwealth and state bodies placed a lot of emphasis on the 
materials that had been developed and made available on the web, others sounded a cautionary note. 

No matter how good the documents are they can never take the place of interactive 
workshops and structured meetings. 

Time 

A new user needs time to become competent in the NRS. Even a well-qualified practitioner needs 
to use the NRS for several months (with support). Everyone interviewed told a similar story: 

It took a long time to get comfortable with it, but now it’s second nature! 

When the NRS was first introduced everyone hated it. There was so much else to deal with at 
that time, and it takes so long to get your head round this as well as everything else. 

If I hadn’t had to learn how to use it I never would have made the effort. I had too much to 
do as it was! 
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Those consulted reported that, with increased workloads, it was becoming increasingly difficult to 
find time to explore the NRS, or to provide assistance to others, and one potential new user 
lamented: 

It takes too much of your own (unpaid) time to work out what it’s about. 



Discussion 

Level of support 

Responses to this survey suggest that the NRS has built a strong following amongst those with the 
background knowledge and opportunity to become expert users. Although in need of an update, 
the conceptual underpinnings were generally considered sound by those academics interviewed. 
Practitioners who had managed to get past its initial conceptual and logistical complexity reported 
the NRS to be an extremely useful tool for gauging a student’s strengths and weaknesses, for 
planning training, and for designing appropriate assessment materials. Those who had used it for 
curriculum development or evaluation had also found it helpful. Many interviewees wondered how 
they had ever managed without it. 

As noted previously, as a tool for reporting to funding bodies, responses were mixed. Those 
involved with the LLNP appeared to be more comfortable with the NRS because they felt it 
contributed directly to teaching and learning outcomes. Those involved with WELL were less 
satisfied, possibly because it was not perceived to help learners, but rather seen as a somewhat 
onerous task to ensure funding continued for a worthwhile program. 

Breadth of application 

Despite concerns about the adoption of the NRS, this study suggests that it has had a wider impact 
than might at first be obvious. 

The NRS is being used directly by literacy and numeracy practitioners within the LLNP and WELL 
programs. It is also being used by specialists in some workplaces not involved with WELL and 
LLNP to inform the design of accredited training, and as part of organisational moves to build 
employees’ language, literacy and numeracy skills. 

The NRS appears to have become the accepted mechanism for ensuring that adult literacy and 
numeracy curricula have the appropriate breadth and depth, and acknowledge developmental 
sequences, levels of support and learning strategies. All major adult language, literacy and numeracy 
curricula have been mapped against the NRS. 

In a field that appears to be wedded to locally produced curriculum this is the most convenient way 
of providing a kind of standardised national accreditation process that is not narrowly prescriptive. 
However, as the curriculum evaluation process does not occur often, the impact of the NRS in this 
area may have been downplayed. The usefulness of the NRS in this role may need to be better 
promoted as part of future curriculum reviews. 

The NRS is being used to inform language, literacy and numeracy aspects of training packages, even 
if there are questions about the effectiveness of interpretation and impact on the quality of literacy 
and numeracy provision in workplace training. 

However, there is concern that it is not being taken up by practitioners who are not formally 
required to use it. 
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Barriers to uptake 

Despite the current breadth of application, most contributors to this project felt that uptake by 
practitioners and systems could be greater. Barriers to take-up identified included: 

•y" The NRS has no ‘system legitimacy’. There is no widespread requirement to use it. 

•y" Practitioners will only make the effort to use it when they can see a benefit for themselves and 
for their students. 

•y" Some practitioners will only make the effort to learn it if compelled to do so. Many of those who 
operate outside the LLNP and WELL cannot see a good reason to do this. 

■v- It is too much trouble to learn. It is hard to ‘get into’. It is not easy to understand. 

•y" NRS resources are scattered or hard to get, but even if they were more accessible, trying to gain 
an understanding of the NRS from documents alone is difficult. 

“v" There is not enough NRS professional development available to help people develop their 
proficiency. 

Need for a review 

Despite the level of support for the NRS identified by this initial survey, there was also an almost 
unanimous belief that it was time for a review. 

I’m worried it isn’t firing people up. So much went into it. We must revisit it to see what we 
can do. 

It is important to note that only one interviewee believed that this should involve a complete 
re- conceptualisation. 

Current understanding suggests that an ICT [information communication technologies] 
approach cannot simply be ‘bolted on’. The whole framework needs revisiting/rethinking if it 
is to reflect literacy and numeracy in the current workplace. 

Most felt the foundations were solid, and would be able to incorporate aspects such as new 
technology. 

There is a need for an interpretive and critical element . . . However, an enormous amount of 
work has gone into it. It would be bizarre to just throw it out. 

It was suggested that the conceptual base be streamlined to produce a simpler instrument that 
retained the ability to capture the richness and complexity of language, literacy and numeracy across 
contexts. The intention would be to strengthen the NRS by drawing on what has been learnt 
through eight years of application, and by incorporating new knowledge and understandings about 
literacy and numeracy, not only as applied to adults, but across all education sectors. 

Keep the open construct and the coherence of the theory, but add aspects. 

Use what is already there, informed by the new multilingual orientation (but with a sense of 
practical constraints). 

A review would also provide the basis for the preparation of user-friendly manuals and other 
resources aimed at literacy and numeracy practitioners, and this in itself could reduce some of the 
current barriers to uptake. Several believed a review could also provide the basis for a ‘public 
conversation’ on literacy and numeracy theory and practice that would be very healthy. 

Try to use its pedagogical capabilities. Anything that informs people about the complexities of 
language is good. 

There were several people who wondered if a review would open up old tensions. However, others 
who had been involved in the original NRS development process, felt that many things had 
changed since that time, and that the climate was more conducive to a constructive review. They 
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also felt that there was no way of avoiding a review, and that the NRS would not be able to sustain 
itself without it. 

It was ahead of its time and therefore it’s still not ‘old’. But it needs updating. You can’t keep 
something like this alive for 10 years without revisiting it. 
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Future directions 



Key questions 

In considering the way forward, the situation can be considered from two different perspectives. 

One option is to focus on the NRS itself and say: ‘Well, we have this instrument that we know is 
actually very powerful. Should we be maintaining it as is, improving it, extending its use or finding 
new applications for it?’ 

In this case, there are three key questions: 

•y" Is the current situation satisfactory? 

•y" If not, what are the problems we should seek to address? 

•y" If we do change anything, what goal do we want to achieve? 

The second option moves the focus away from the NRS itself and onto the broader context of adult 
language, literacy and numeracy. 

In this case, we need to ask: 

■v" Where are the areas of greatest need in adult literacy and numeracy provision? 

•y" Could the NRS (in its current role and guise, or in a new one) be useful in addressing any of 
these needs? 

The most appropriate options are likely to be those which meet an identified priority in the broader 
context. Insights into the way this might happen can be found by considering the inherent strengths 
of the existing instrument and learning from the ways in which it has been applied to date. 

Questions pertaining to both perspectives are explored further below, the discussion incorporating 
relevant input from interviewees as outlined in earlier sections. 



Exploring options for the NRS 

What is the real problem? 

Many of those interviewed identified what they saw as ‘the real problem’ confronting the NRS. 
There is no problem. Leave things as they are. 

The real problem is that the NRS is not easy to understand. 

The real problem is that the NRS is not being used by enough people. 

The real problem is that the NRS is not in fact a national reporting system. 

The real problem is that the NRS is a wasted resource. It is not just for reporting and needs to 
be recognised, and used, as a multi-purpose tool. 

The real problem is that the NRS is not formally part of the mainstream of adult education 
and training. 
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In setting future directions, the perception of the key problem will affect the choice of goals and 
strategies setting the tone of future development. Therefore each option is explored further below 
with reference to interviewees’ responses. Table 2 at the end of this chapter provides an overview 
and identifies potential consequences of each action, both intended and unintended. 



Discussion 

There is no problem. Leave things as they are 

Potential goal: Maintain the status quo. 

No one actually wanted to leave the NRS instrument exactly as it was. All interviewees felt the 
NRS was overdue for revision, and were unanimous in calling for a review. However, almost all 
wanted to build on the existing contextual framework, not replace it. Although there was some 
potential to stir up philosophical and pedagogical disagreements, those interviewees who had been 
privy to the animosity surrounding the birth of the NRS felt that a review was unlikely to cause the 
same divisions. Despite the risks, a critique could give the NRS a new legitimacy, and the 
discussions involved could be a vehicle for professional development, while also raising the profile 
of the NRS itself. 

Some interviewees did want to maintain the status quo in terms of limiting the NRS to its role as a 
reporting mechanism, and could not envisage its being used for any other purpose. Others felt that 
the NRS was already being used in many different ways — but that there needed to be greater 
recognition of this, perhaps through repackaging and a name change that better reflected its multi- 
purpose nature. 

The real problem is that the NRS is not easy to understand or use 

Potential goal: Make the NRS more user-friendly and accessible. 

A major criticism of the NRS is that it is ‘not easy to get your head around’. Even experienced 
practitioners must invest time and effort just to understand it, let alone to become competent users. 

It does not help that the original manual is ‘daunting’, or that support materials can be hard to 
track down. 

Almost all interviewees felt that the NRS could be streamlined conceptually without undermining 
its ability to manage the complexity of factors associated with language, literacy and numeracy 
competence. Most were keen to improve the NRS by incorporating new academic knowledge and 
drawing on the knowledge of experienced users. In particular, the LLNP moderation/verification 
process could provide a wealth of data and many insights into what works and what doesn’t and why. 

Interviewees made a distinction between streamlining and simplifying. There should be no attempt 
to ‘dumb the instrument down’ just so that non-specialists could use it as a reporting tool. What 
would be the point anyway? However, there was potential to make key NRS concepts more 
accessible to non-specialists such as workplace trainers. Indeed, several simple ‘versions’ have already 
been developed and are reportedly proving effective in this regard. 

Another suggestion was that the NRS could be revised and presented as a type of training package 
in its own right, or that it could be explained using headings and terms used in competency-based 
training. This alignment with what is now ‘mainstream’ would make it more accessible to both 
specialists and non-specialists. 

Even though the print and web-based materials need to be revisited and made more user-friendly, 
interviewees were adamant that any attempt to make the NRS more accessible to specialists must 
involve a recognition that interactive professional development is the key to initial understanding of 
the NRS conceptually, and to the development of expertise in its use. As new specialists do not 
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necessarily have access to NRS workshops, current resourcing of professional development should 
be reconsidered as a matter of urgency, or the NRS will founder. 

The real problem is that the NRS is not used by enough people 

Potential goal: Increase uptake of the NRS in order to improve practice. 

A minority of interviewees believed that the NRS has had only a limited effect on adult literacy and 
numeracy, and saw this as an indictment of the tool itself, or of the system. However, there is 
evidence to suggest that the NRS has actually had a greater impact than is immediately apparent. It 
may be that it has not been heavily promoted. People are not actually aware of the degree of 
influence it has had on the practice of specialists around the country, or of how far it has informed 
curriculum development and evaluation, or of its role in the design of training packages. 

It all depends on what counts as ‘uptake’. It may be useful to make a distinction between specialists 
and non-specialists. 

If the aim is to increase the number of specialist users adopting the NRS, it would be helpful to 
have a better understanding of the current breadth and depth of uptake in the profession, and some 
insight into what prevents some practitioners from using the NRS. (Is it, as some interviewees 
claim, simply too hard? Is it lack of time or support to learn it? Is it lack of understanding of the 
potential of the NRS to help them in their work?) It might then be possible to design strategies to 
raise the profile of the NRS amongst specialist non-users and encourage them to try it. 

Interviewees generally agreed that the NRS in its current form is not a reporting tool for the non- 
specialist, and few argued that addressing this situation should be a future goal. However, if the 
ultimate goal of any future NRS strategy is to influence practice, there could be a place for a version 
of the NRS that captures its key concepts. 

This could be used as a tool to raise awareness about language, literacy and numeracy issues relevant 
to a workplace training or community context. It could be incorporated into professional 
development activities to give people across the VET or ACE sectors a nodding acquaintance with 
critical language, literacy and numeracy concepts. It would also increase the mainstream legitimacy 
of the NRS. 

However, the existence and widespread promotion of a simplified NRS alongside a full NRS has the 
potential to create confusion. 

The real problem is that the NRS is not in fact a national reporting system 

Potential goal: Push for national uptake as a reporting mechanism. 

There was little support for this as a way forward. A number of interviewees were happy that the 
NRS was used within LLNP and WELL if only because this kept it alive, but no one was keen to 
see it mandated for reporting elsewhere. Only one practitioner thought it should be national for the 
sake of consistency, and was concerned that transportability only worked if a student went from one 
LLNP to provider. Others argued against any push for a single national reporting system on the 
basis that total consistency would stunt growth by removing the current diversity and potential for 
cross-fertilisation. 

At least one state training authority had rejected the idea of adopting the NRS, believing it would 
not achieve any useful outcome and could in fact do some harm if information was misused. 

There were also concerns about many reporting requirements within the VET sector. Adding the 
NRS would not actually achieve anything other than practitioner overload and anger. 

There was a close connection in some people’s minds between making the NRS national (and 
therefore compulsory) for reporting purposes, and a move to bring in one adult language, literacy 
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and numeracy curriculum. There was strong resistance to the latter, with most interviewees 
preferring the current situation, with several different curricula and state-based programs which 
could be compared and evaluated as required using the NRS. Any move to nationalise the NRS 
would need to anticipate and address this potential misinterpretation. It would also need to counter 
the immediate resistance that accompanies any bureaucratically imposed compulsory requirement. 

The sheer energy required to push for a single mandated reporting process, and the degree of 
resistance this is likely to produce, suggests that this is not a useful pathway, unless some genuinely 
useful purpose is found to justify the collection of data. One possible avenue of investigation might 
be in relation to data for comparison with overseas International Adult Literacy Survey data. (The 
NRS was originally designed to align with the International Adult Literacy Survey although it 
actually provides much broader information.) 

Interestingly, the flexibility of the NRS and its application in curriculum mapping and training 
packages may mean that it is already in a position to achieve at least some of the goals of a national 
system without actually being ‘national’. 

The real problem is that the NRS is a wasted resource. It is not just for 
reporting and needs to be recognised and used as a multi-purpose tool 
Possible goal: Reframe and reposition the NRS. 

Possible strategy: Use as the basis for the development of an adult literacy and numeracy framework 
or profiles. 

Some interviewees were asked about the idea of using the NRS structure and concepts as a 
framework for describing and analysing language and literacy, or as the basis for the development of 
a set of Australian literacy and numeracy profiles with levels and descriptors of behaviour and skills. 

Some rejected the suggestions, feeling that the NRS had to remain true to its original 
purpose — which was primarily as a reporting tool. They argued that conceptually it could not ever 
be anything else. If it were redesigned for other purposes, it would not be the NRS anymore. Several 
interviewees were worried about introducing yet another ‘framework’, referring back to an 
‘Australian Literacy and Numeracy Framework’ which predated the NRS and contributed to it, but 
was never used extensively in its own right. However, they seemed to think it could not be replaced. 

Not surprisingly, those who were actually using the NRS in a variety of ways disagreed. They were 
particularly supportive of the role of the NRS in professional development, and as a curriculum 
framework, and were interested in ways of supporting these existing applications more extensively. 
However, they identified difficulties associated with promoting the National Reporting System as a 
multi-purpose tool, if only because the name suggests such a specific purpose. 

Most interviewees felt that one of the most important contributions of the NRS was the provision 
of a common framework and language for talking about and analysing language, literacy and 
numeracy within the profession. It would not take a great deal to rework its elements as a formal 
literacy and numeracy framework to replace the one that had gone before. A new framework could 
fill this role and provide the basis for development of versions of varying complexity without 
compromising the most detailed and sophisticated version, which would still be used for purposes 
such as formal reporting and curriculum evaluation. 

The key would be that the central constructs of the framework would align across all versions, and 
provide a bridge and a common language from specialist to workplace trainer to volunteer to 
employer. 

Most were more interested in the idea of a framework than they were in that of ‘profiles’. However, 
there were very different interpretations of what ‘profiles’ meant, so this may not be an indication of 
resistance per se. If discussions are to continue in this direction, the terms profile and framework will 
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need to be used cautiously, and time invested in developing the concepts without necessarily 
labelling them in advance. 

The real problem is that the NRS is not formally part of the mainstream of 
adult education and training 

Possible goal: Find a way of making the NRS ‘part of the way we do things round here’. 

The importance of language, literacy and numeracy is increasing within the VET sector, at least in 
theory, with recent ANTA documents stating that: 

The National Strategy for VET recognises that language, literacy and numeracy has to be here 
there and everywhere. (ANTA 2003, p.6) 

Does this mean that the NRS should also be all-pervasive? 

Discussions are currently occurring about how best to ensure that literacy and numeracy are 
addressed adequately within vocational education and training. A review of the NRS in this context 
may identify ways in which it could contribute to better understanding of language, literacy and 
numeracy. As a starting point, interviewees were asked to consider two different strategies for 
linking the NRS to the mainstream. 

V Incorporate the NRS into ‘employability skills’. 

V Position the NRS as a means to bring literacy and numeracy ‘upfront and central’ within revised 
training packages. 

Incorporate into employability skills 

LL&N competence underpins employability skills. Clearly the ability to use technology, 
manage change, communicate and work constructively with people is underpinned by a 
plethora of speaking, listening, reading, writing and numeracy practices. (ANTA 2003) 

Phis concept of literacy and numeracy as enabling skills underpinning employability skills was 
embraced by the developers of the Australian Chamber of Commerce and Industry and Business 
Council of Australia employability skills, but they rejected the idea of incorporating language, 
literacy and numeracy directly. 

With literacy and numeracy sitting ‘behind’ or ‘beneath’ the employability skills, could the NRS 
provide an explicit link? 

A study by Walker (1996) determined that each of the seven Mayer Key Competencies could be 
mapped to the NRS competencies and learning strategies, although there is no neat alignment of 
the numbers. (Key competency performance level 1 equates to NRS levels 2 and 3, performance 
level 2 to NRS level 4 and performance level 3 to NRS level 5.) Although experienced practitioners 
can appreciate how the key competencies and NRS connect, there is no simple way of 
demonstrating this to others. A similar situation exists with the more recently developed 
employability skills. 

Some interviewees were unsure of the usefulness of attempting to combine the NRS and 
employability skills, citing the confusion between different sets of ‘levels’, the potential for over- 
simplification and the accompanying loss of detail that currently gives the NRS its power. 

There was also a broader discussion of the potential damage that could occur if literacy and 
numeracy were in fact treated as ‘generic’ skills. 

Would this undermine the message about the importance of context and reinforce a concept 
that literacy and numeracy are basic skills and entirely transferable? 
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Several interviewees were concerned that trying to subsume literacy and numeracy under the 
heading of ‘employability skills’, would narrow the focus of language, literacy and numeracy 
teaching to the workplace. Others criticised this stance as naive, citing research that demonstrated a 
clear connection between literacy and numeracy performance, employability, effective functioning 
in life and personal self-esteem. 

The NRS and training packages 

The high-level review of training packages (Schofield & McDonald 2004, p. 1 5) identifies literacy 
and numeracy education as among ‘specific needs not directly connected to a job outcome which 
are either not amenable to or fully covered by a training package approach’. 

However, this review also recommends that generic skills be ‘front and centre’ in revised training 
packages. Although it is not made clear, there is an implication that the term generic is being used 
more broadly than within the Mayer Key Competencies or employability skills, and includes 
literacy and numeracy. 

As previously discussed, a number of interviewees were concerned about the status of literacy and 
numeracy within training packages, and as approached within workplace training in practice. 

Several saw a role for the NRS — or the concepts it encapsulates — in helping to raise understanding 
of language, literacy and numeracy within workplace training. They believed it could provide the 
common framework and language to support the discussions about language, literacy and numeracy 
that obviously need to occur. A further suggestion was that specific training package units could be 
developed based on NRS levels, or that much more specific advice and training should be given to 
designers to ensure that appropriate NRS levels were assigned to language, literacy and numeracy- 
rich units. 

However, a major stumbling block is that the NRS is not particularly accessible in its current form. 
Even if designers get specific and extensive training, will this have much impact on workplace 
trainers and training? Could a simpler version of the NRS (or better still a new LLN framework), 
provide a way of raising awareness for them without undermining the role of the full NRS as a 
reporting tool for specialists? 

Even allowing for the fact that training packages are ‘not synonymous with VET provision’ 
(Schofield & McDonald 2004), and that the VET sector does not embrace all those in need of 
adult literacy and numeracy assistance, a version of the NRS could be linked more prominently to 
both the design and delivery of training packages. This would ensure that key concepts and 
messages become part of mainstream understanding. 

A purpose for change 

Each aim and strategy outlined above had supporters and detractors, but it should be noted that 
when any of these options was put to those interviewed, a common response was a pause, followed 
by a comment like, ‘Well I suppose you could, but why would you?’ 

This highlights the piece that is currently missing in this picture: ‘What is the purpose of change?’ 
There is still a decision to be made in this regard, and the question provides a lead-in to a 
consideration of the context within which the NRS operates. 

In charting a way forward, it is useful to identify needs and priorities in adult literacy and numeracy 
first and then consider whether the NRS, in some form, could support the achievement of broader 
goals. This would provide the raison d’etre for any significant reworking. This perspective is 
discussed below. 
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Priorities and needs in the field 



The NRS cannot be viewed in isolation. In considering future directions, it needs to be placed not 
only within the adult literacy and numeracy field, but within the broad context of adult education 
and training and Australian social and workforce development. This is a project in itself. Thus, the 
following discussion is necessarily brief, and focuses mainly on the relevant input of contributors to 
this project. However, their comments provide useful insights into issues that may need to be taken 
into account in deciding where and how the NRS might assist with the achievement of national 
priorities. 

Crucial issues 

In designing future directions for the NRS, the following issues should be considered: 

•T the current lack of a high-level strategy for language, literacy and numeracy 

•T formal endorsement of the need to integrate language, literacy and numeracy within general 
adult education and training 

■y" the recommendations of the high-level training package review 
•y" the blurring of the distinction between the schooling sector and VET 
•y" the influence of overseas strategies and priorities 

•y" characteristics of, and trends affecting, the adult literacy and numeracy workforce. 

Discussion 

Crucial issues can be seen as stumbling blocks or barriers, but they can also provide indicators of 
potential opportunities. 

The NRS and the development of a new literacy and numeracy strategy 

The NRS will never be widely used if it remains, in effect, a stand-alone mechanism. However, it 
cannot be linked effectively to other strategies to achieve agreed high-level goals because there is 
currently no overarching strategy or set of agreed priorities to give the current (or revamped) NRS a 
national purpose. Nor does there appear to be any one body able to take responsibility for driving 
high-level change. 

There are moves to frame a new national strategy for adult language, literacy and numeracy. This 
has grassroots support, and may be given increased political impetus if the results of the planned 
2006 International Adult Literacy Survey show that Australia has made little progress in addressing 
adult literacy and numeracy issues. Poor results in the last survey galvanised the United Kingdom 
into action and a similar situation may occur here. Other countries are placing greater importance 
on (and increasing resources to support) adult literacy and numeracy, and this too may have a 
political impact in Australia. 

Discussions about language, literacy and numeracy provision go beyond the NRS, but within these 
conversations, it will be important to ask how the NRS might be utilised within a renewed effort to 
raise the levels of adult literacy and numeracy competence in Australia. This could be as a formal 
reporting tool, as part of a professional development program, as a recognised curriculum 
framework, or as a basis for literacy and numeracy profiles. It all depends on the priorities identified 
within a high-level strategy. 

It may take some time for such a strategy to be developed and endorsed, but this should not 
preclude a review and revision of the NRS, its reworking as a reporting tool, or its continued use in 
non-reporting applications. However, as long as there is no strategy, it may be difficult to make a 
case for funding for major professional development programs, incorporating the NRS. 
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Link to the professional development of the literacy and numeracy workforce 

A grand plan for language, literacy and numeracy is needed, but even if it is developed, it will come 
to nothing without people with the skills and knowledge to make it happen on the ground. 
Indicators are that there is a growing need to develop the capacity of the language, literacy and 
numeracy workforce. 

V The majority of those in the professional language, literacy and numeracy workforce are ageing, 
and there are few young people entering a profession that offers a limited career path, low pay 
and the uncertainty associated with sessional work. 

V Formal professional development is limited, and heavy work demands, casualisation and high 
staff turnover in some contexts mean few opportunities to talk professionally with peers. 

V With few new entrants to post-graduate courses, and a reduction in the availability of such 
courses, the trend is towards language, literacy and numeracy qualifications requiring less formal 
study and of lower academic status. 

“v" A recent plan to increase the level of language, literacy and numeracy training in the new 
Training and Assessment Training Package was rejected by professional bodies resisting the 
‘lowering of standards’ so one potential avenue for language, literacy and numeracy professional 
development had been minimised. 

Clearly, the maintenance of a trained workforce requires a raft of integrated strategies, but one of 
these could be the provision of professional development workshops for members of the following 
groups. Whether they are the ‘right’ people for the job or not, they may well be increasingly 
responsible for improving adult language, literacy and numeracy skills: 

L professional practitioners working as specialists in language, literacy and numeracy fields. They 
have post-graduate qualifications and may be involved in delivering accredited curriculum in 
adult basic education 

V language, literacy and numeracy practitioners (often ex-teachers) with some training in language, 
literacy and numeracy (for example, Certificate IV in Literacy) 

L vocational teachers with little or no background who are responsible for managing the language, 
literacy and numeracy requirements and needs within vocational certificates in their fields 

V community literacy providers who may be volunteers with limited formal training (for example. 
Volunteer Tutoring Certificate). 

These workshops would encompass more than the NRS, and provide an appropriate context within 
which to raise awareness of its concepts and applications. It becomes easier to envisage this having 
an impact if NRS concepts were to be translated into a general literacy and numeracy framework. 

Link the NRS to training package reform 

Training packages are now an accepted part of the VET sector. The importance of raising the 
language, literacy and numeracy awareness of designers and workplace trainers has been 
acknowledged for some time, but more as a theory than as an effective practice. In light of 
experience, a strategy to increase competence will have to go much further than a requirement for 
developers to take notice of the NRS. Once again, a national language, literacy and numeracy 
framework has the potential to begin a process of change by making key concepts accessible and by 
providing consistent standards and a common language for industry members, workplace trainers 
and specialist practitioners. 

Use the NRS to cross-educational boundaries 

There is little alignment between school-based approaches to literacy and numeracy and those 
adopted in vocational education and training. Worrying numbers of young people are leaving 
school early with low levels of literacy and numeracy skills, and many are turning to ACE for 
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assistance. This is raising questions about funding responsibility and accountability: the schools 
sector is funded to support these young people, while the ACE sector is not. 

Does the schooling sector need to learn new approaches from those in adult literacy and numeracy? 
Is there a role for the NRS to help cross the divide? Some interviewees were keen to develop this 
idea further, and it is interesting that the potential of the NRS to provide a bridge has recently been 
raised in at least one state. On the other hand, the schooling sector has learnt a lot about 
conceptualising, presenting and implementing profiles, and this could be useful. Some adult literacy 
and numeracy experts already cross the divide regularly, and successfully, and expressed interest in 
assisting with the cross-fertilisation process. However, any mention of the potential to learn from 
the schools sector was greeted with derision by most interviewees. This is not necessarily a reason 
not to pursue this further, but such a strategy might need to be addressed carefully! 



Table 2: Overview of possible goals, strategies and outcomes 



Potential goals Possible strategies Outcomes 

If the goal is to ... We could... This may lead to intended and unintended 

consequences, such as .... 



1 Maintain the status Leave things as they are. 
quo. 



2 Increase uptake of Make the NRS more accessible and 
the NRS. user-friendly for specialists. 

Streamline the instrument. 

Revise materials and simplify 
presentation for specialists. 

Develop appropriate professional 
development as major vehicle for 
increasing understanding and 
proficiency. 



NRS maintains low profile amongst non-users. 

NRS use gradually falls away as ageing practitioners 
retire and NRS professional development remains 
difficult to access. 

NRS fails to keep up with changes in understanding, 
expectations re literacy and numeracy, technology 
etc. 

NRS formally reviewed as basis for revision. Better 
understanding of what works/doesn’t work and why. 

Development process becomes vehicle to encourage 
discussion about language, literacy and numeracy 
across the field. 

Updated, relevant instrument and materials. 
Increased understanding of and respect for NRS. 
Reduced psychological barriers for new users. 

Potential to build capacity of profession through high- 
quality professional development that incorporates 
NRS. 

Potential to interest current non-users. 

Potential to start new controversies if not well 
managed. 



...for non-specialists Design and promote simplified version 
for non-practitioners. 

Promote through professional 
development. 



3 Push for national 
uptake as a 
reporting 
mechanism. 



Persuade states/territories to 
implement. 

Mandate for all Commonwealth 
programs. 



Increased understanding of key concepts re literacy 
and numeracy. 

Increased awareness of when and why to get 
specialist help. 

Increased awareness of/legitimacy for NRS in 
broader VET community. 

Standard reporting format to give national 
consistency. 

Widespread appreciation of NRS framework. 
Potential to compare range of language, literacy and 
numeracy-related aspects across ACE and VET. 
High-profile discussions of NRS. 

Resistance to interference; conflict, rekindling of 
previous disputes, even refusal to use NRS at all. 
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Potential goals Possible strategies 

If the goal is to ... We could... This may lead to intended and unintended 

consequences, such as .... 



Outcomes 



4 Reframe and Use as basis for development of 

reposition the NRS. Australian literacy framework with 
multiple applications. 



5 Link the NRS into Position NRS as part of major push to 
the mainstream. make language, literacy and numeracy 
‘upfront and central’ in training 
packages. 



Incorporate into employability skills. 



New name, image and concept that better 
encapsulates the multi-purpose nature of the NRS. 
Increased acknowledgement of non-reporting 
applications. 

Vehicle for promoting professional discussion and 
reflection. 

Confusion? Loss of ‘brand’ loyalty? 

Increased understanding amongst non-specialists of 
language, literacy and numeracy complexity within 
workplace. 

High profile for NRS as framework for discussing, 
analysing, benchmarking. 

Years of argument and frustration trying to get initial 
agreement. 

One set of essential skills. 



Confusion over generic levels vs need to 
contextualise language. 
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Conclusions and recommendations 



Conclusions 

Degree of impact and level of support 

When the NRS was first released, one of the NRS developers reported feeling that it would survive 
if it was actually good enough. Despite widespread initial resistance, the hiatus that followed a 
change of government, the cancellation of programs and the lack of planned professional 
development, indications from this preliminary survey suggest that it has indeed survived, and 
proved its worth. 

The NRS has provided feedback on the impact of language, literacy and numeracy funding — as was 
originally intended. Although there is a strong perception that it has not had the impact it should 
have done, there are indications that it is more firmly entrenched within the adult language, literacy 
and numeracy context than might at first appear. It has the support of some prominent and 
influential people in the field, and has become an integral part of the philosophy and methodology 
of some expert and highly regarded practitioners. It sits quietly behind the major adult language, 
literacy and numeracy curricula, and will play a role in mapping these again as they come up for 
re-accreditation. The NRS has also been integrated into a limited number of workplaces in its own 
right, while having an indirect influence through its role in WELL. 

Time for review 

Although once recognised as ahead of its time, the instrument now needs updating, particularly in 
light of developments in information and communication technology. It also needs streamlining 
conceptually. Although it may well have been a strategic move to give the NRS time to bed down, it 
is now timely to open it up for review. There is real interest. People want to talk about it. People 
want to critique it. Although there are doubtless those opposed to the NRS who were not 
interviewed, this preliminary survey suggests that the process could be stimulating for those in the 
field if it is specifically designed to facilitate constructive exploration and sharing of ideas. 

There is no point in going ahead with the development of new applications for the NRS until a 
review has been completed. However, discussion of future directions and possibilities should be 
incorporated into the professional conversation. 

The new NRS: Concepts + materials + professional development 

A revision reflecting input from many expert users would provide a more accessible user-friendly 
tool and incorporate the best of the materials already developed. A revision should also involve a 
review of the rules for reporting within Commonwealth-funded programs to ensure that they do 
not inadvertently undermine the spirit of the NRS. At the very least, the process should produce a 
reporting tool that would make life easier for newcomers to the Language, Literacy and Numeracy 
and the Workplace English Language and Literacy programs. This in itself would be a valuable 
outcome in light of high staff turnover in some quarters, the lack of mentors, and the trend towards 
a lower qualified workforce generally — all of which may have an impact on the quality of NRS 
application in the future. 
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However, no matter how streamlined a new NRS may be, over-reliance on the value of print and 
web-based materials could still undermine uptake and effective application. The NRS will never be 
genuinely accessible — and will never therefore have the level of impact on practice that it could — 
without interactive professional development. This should be perceived and resourced as an integral 
part of any NRS ‘package’. 

Specialists in the field are desperate for professional development. They want the opportunity to 
come together to share ideas, and learn. Workshops introducing a revitalised NRS as a framework 
for analysing language, literacy and numeracy (as opposed to a focus on how to use the NRS to 
report to funding bodies) could meet this priority need. 

Built in, not bolted on 

Although the NRS has gradually become interwoven into the fabric of adult literacy, it does not 
have a high profile, and has not become an integral part of mainstream adult education, despite 
recognition of the need to increase the literacy and numeracy understanding and skills within the 
VET sector. However, the concepts encapsulated within the NRS could be extremely useful in this 
regard, and could be promoted as part of the revision of training packages, where language, literacy 
and numeracy will doubtless play a prominent role. However, even a revitalised NRS is likely to 
present as ‘too much information’. A literacy and numeracy framework, based on key NRS 
concepts, would provide an effective awareness-raising tool. It would also be easier to promote to 
non-practitioners, and would be more likely to be adopted in practice. 

More than a reporting tool 

Since its implementation, the NRS has been ‘more than a reporting tool’. The revision of the NRS 
should involve the exploration of the potential for a new framework for thinking, analysing and 
discussing language, literacy and numeracy in multiple contexts. 

Despite some people’s reservations about the term, a framework provides the scaffolding upon 
which many things can be built. The NRS has already shown that it is flexible enough to become 
the support structure for multiple purposes. Developing a new entity, and calling it a literacy and 
numeracy framework, would signal a formal recognition of multiple applications in a way that has 
not been possible with the NRS. It would be a more logical step than continuing to try to explain 
how a reporting mechanism can somehow actually be useful for many other purposes as well (an 
extremely convoluted process at the moment). The very fact that it is a recognised framework 
should make it clear that it can be used to underpin a range of different applications. 

The new framework could be developed in several formats for various purposes and audiences. A 
full (revised) version would still offer the richness of the original NRS, while a simplified version 
might be used to introduce key concepts about literacy and numeracy to non-specialist audiences, 
and thus become an important tool in the push to improve general workplace understanding of 
language, literacy and numeracy. The revised NRS itself could still be used for reporting purposes, 
and remain the ‘national reporting system'. Despite current resistance to the idea, there may also be 
a place for the development of a broader national reporting system to provide relevant data on adult 
literacy and numeracy performance and provision, and using the NRS as a tool. 

Even though there might be different versions of the framework, the crucial element would be that 
they would all be built on the same foundations, using the same terms and levels. Besides 
maintaining consistency, this in itself would build greater understanding and appreciation of 
language, literacy and numeracy concepts and issues, and provide a link between specialists, 
workplace trainers and industry. The formal launch of such a framework could be a vehicle for 
much needed capacity building within the adult literacy and numeracy field. 

A literacy and numeracy framework might also be more easily placed ‘behind’ the employability 
skills to show what the enabling skills of language, literacy and numeracy look like, and how they 
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may impact on an individual’s competency levels in, for example, communication or problem- 
solving. However, this would still need to be managed carefully to avoid confusion over differences 
in numbers of levels and degrees of detail. 

The role of the NRS as a reporting mechanism should not be discounted in a push to broaden its 
application or appeal. It still has an important part to play in providing useful data to funding bodies, 
and in demonstrating the positive impact of training provision on individuals and workplaces. 



Figure 2: The roles of an Australian literacy and numeracy framework 
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Recommendations 

■v" Continue to use the NRS as a key tool in reporting on adult literacy and numeracy performance. 

■v- Review and revise the current NRS. Develop a package incorporating the instrument itself, a 
revised set of rules for use for reporting within Commonwealth programs, a set of user-friendly 
support materials, and professional development activities designed for specialist users. Design 
the review process as a vehicle for professional development in its own right. Use it to further 
explore a range of options for future directions for the NRS and build a sense of ownership of 
the outcomes. 

■v - Explore the potential to use the key concepts and structure underpinning a revised NRS as the 
basis for the development of a literacy and numeracy framework (with a new name) which could 
be presented in different formats and levels of complexity for a range of audiences and purposes. 
This option could also be incorporated into the proposed review. 
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■v" Review the arrangement for provision of NRS training and ongoing mentoring within the 
LLNP and WELL. Consider ways of funding and delivering ongoing language, literacy and 
numeracy training to other target groups, for example, through Industry Training Councils and 
professional bodies. 

^ If a new language, literacy and numeracy framework is developed, promote it through 

interactive professional development programs tailored for different groups to enhance their 
understanding of language, literacy and numeracy. 

•y" Devise a strategy to ensure that a new framework (or failing that, the revised NRS) plays a key 
role in determining the language, literacy and numeracy aspects of revised training packages. 
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Appendix 2 
NRS Question Framework 



A. Background 



A1 


Name 


A2 


State 


A3 


Current position/role 


A4 


Literacy/numeracy background; years of experience, qualifications. With which groups? 


A5 


Experience with NRS - settings, # years 


A6 


Drivers. Why did you start using the NRS? or barriers. Why don’t you use the NRS? 


A7 


NRS training. How did you learn how to use the NRS? 



B. The NRS Instrument 



B1 


What is your understanding of what the NRS is for, and who is it for? 


B2 


Is it appropriate for this (these) purposes(s)? i.e. Does it do this job? 


B3 


How do you (have you) used it? 


B4 


What do you see as its strengths? 


B5 


Issues with use? 


B6 


Is it theoretically sound? Relevant? Need for critique? 


B7. 


Connection to/comparison with IALS? 


B8 


Does it do any good? Any harm? 



C. The NRS as a Formal Reporting Mechanism 



Cl 


Background knowledge and experience in use of NRS in this context? 


C2 


Strengths of current arrangements? 


C3 


Issues? 


C4 


Impact on various stakeholders - learners, teacher, organisation, funder 


C5 


Who are stakeholders in an effective NRS? Who do you see as the priority stakeholder? Who should be? 


C6 


Resources needed. Time/cost/expertise. 


C7 


How are data used? Is it telling people what they need to know? 


C8 


Is it trusted/trustworthy? Is it fair? Does it do any good? Does it do any harm? 



D. Requirements for effective use of NRS 



D1 


What is needed for current NRS to be effective? 


D2 


What background knowledge and formal qualifications do users need if they are to use the NRS effectively 
for different purposes? Minimum training/support required? 


D3 


Current training specifically in NRS 

What is available in your state? For whom? Accessibility? Frequency? Relevance? 


D4 


Best way to develop an understanding of the NRS? To become proficient? 




Should anyone be able to use it? Why/why not? Implications? 
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E. Non-reporting applications 



El 


Who is using the NRS outside of LLNP, WELL and why? 


E2 


How is it used? 


E3 


Effectiveness for such purposes? Perception of greatest usefulness? 


E4 


Which learner groups does it ‘suit’? Issues for any groups? 


E5 


If not used for a specific group, what tools are used? 



F. NRS: Exploring future applications 



FI 


What do you see as the most important issues and trends in adult literacy and numeracy? 
In adult vocational and/or community education in general? 


F2 


The NRS and training packages or in employability skills 


F3 


The NRS as a national framework for discussing LLN - not specifically as a reporting system at all 
e.g. as set of national profiles. 


F4 


The NRS - potential for different versions for different purposes and audiences 


F5 


Should the NRS be simplified so anyone could use it? 


F6 


Should the NRS be ‘national’? If so what does this mean and how might it be effected? 


F7 


Does it need a name change, and if so, what should it be called? 
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Appendix 3 Snapshots 



The following snapshots provide a comparison of two different practitioner perspectives on the NRS. 

Practitioner operating in the WELL program 

I am not a literacy teacher. I teach communications, not how to read and write, and I run 
somewhat short focused, specialised courses on things like how to run team meetings, or 
improving food safety, not literacy. 

1’his interviewee believes the WELL program itself is invaluable and does make a difference to 
people’s skills. She also thinks it is reasonable for the funding body to expect some accountability, 
but she can see no benefit from using the NRS for this. 

It’s not that it is impossible to use when describing the language and literacy requirements of 
team meetings, but I can’t see the point. It’s just jumping through hoops. 

Although she is quicker now that she is more confident, she reports that it, ‘used to take hours to fill 
in the forms’. She believes it might make more sense if the NRS were used to describe progress, but 
at the same time, doubts that it would be possible to measure much progress over a 20-hour course. 

It doesn’t justify the amount of time it takes to learn how to use it, or the effort you have to 
put in to fill in the forms. 

She can, however, see potential for the NRS within a professional development program as a language 
for discussing people’s skills, but she would rather see more effort put into the sort of reading and 
writing requirements for workplace competencies, and does not see a great role for the NRS in this. 

Her thoughts were echoed by a member of an industry training body who described the use of the 
NRS within WELL as ‘overkill’. 



Numeracy specialist working with learners in the LLNP 

I find the NRS very useful now, but new trainers have a devil of a job with it, and I did too. It 
took 3-4 months to get familiar with it, and I found it tedious having to keep evidence on 
everything. But having to use it meant I was forced to get familiar with it, and now I find it 
does actually help me to see where someone is functioning in each area. 

This practitioner believes one of the keys to effectiveness is making time to write up responses to the 
pre-training assessment as quickly as possible — ‘the further you get from the assessment the more 
difficult it is to keep focused’. She knows that some trainers cannot do this, often because their 
organisations do not have effective processes to support this. She felt her organisation was successful 
because it had good systems in place, having developed its own pre- assessment tasks mapped to the 
NRS, and internal processes to support trainers in keeping good records. However, it still took time 
to do the thinking and the paperwork, and she was currently struggling to find a way to manage her 
increased workload in this regard. Her own efforts in promoting the services of her not-for-profit 
organisation had led to a significant increase in client numbers, but the time available for 
administration had not changed at all. 
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